Performance testing is a type of software testing which ensures that the application is performing well under their expected workload. The goal of performance testing is to eliminate performance bottleneck rather than finding bugs. Features and functionality supported by a software system is not only the concern so as, used to determine, measure, validate or verify quality attributes of system and application like speed, scalability, stability and responsiveness under variety of load condition. Run performance tests when you want to check your website and app performance, which may extend to testing servers, databases, networks, etc.
What is Load Testing in Performance Testing
Testing conducted to understand the behavior of an application when large numbers of users access the application simultaneously. It is a non-functional software testing process usually conducted by the performance engineer. The primary goal of load testing is to define the maximum amount of load a system can handle without significant performance degradation. Load testing is commonly used for the client/server, web-based applications.
Objectives of Load Testing
To maximize the operating capacity of a software application. To determine whether the software application capable to run latest infrastructure or not.
Why Load Testing is needed?
Load testing gives confidence, reliability and performance in the system. Helps to determine the operating capacity of an application. Load Testing helps identify the bottlenecks in the system under heavy user stress scenarios before they happen in a live environment.
Load testing gives excellent protection against poor performance and accommodates complementary strategies for performance management and monitoring of a production environment.
When Load Testing should be performed?
Load test when you want to determine how many users your system can actually handle. Some extremely popular sites have suffered serious down times when they get massive traffic volumes. E-commerce websites invest heavily in advertising campaigns, but not in Load Testing to ensure optimal system performance, when that marketing brings in traffic.
How to do Load Testing in Software Testing?
Load testing can also be performed without the help of any load testing tools. The basic steps to do load testing for an application are listed below:
- The first step involves creating a dedicated test environment for load testing.
- Determining the Load test scenarios.
- Then determine the load testing transactions for the application that include:
- Preparing test data for each transaction
- Predicting the number of users accessing the application
- Determining the connection speeds
- Determining the different browsers and operating systems on the user’s end
- Configuring web, application and database servers
- Execute the test scenario and monitor the results by collecting various metrics
- Analyzing the results for making recommendations
- Tuning the system and retesting it
Example of Load Testing
- Running multiple applications on a computer or server simultaneously
- Using thousands of concurrent users testing a mail server
- Testing a printer by transferring a large number of documents for printing
- Downloading a series of movie files from the internet
- When Amazon.com servers crashed in 2013 for 30 minutes, Amazon lost huge amount.
Tools for Load Testing
- HP Load Runner (PAID/LICENSED)
- Triscentis Flood
- Apache Jmeter (open source)
Stress testing is performed to ensure that the system would not crash under crunch situations. It is a non-functional testing and also known as Endurance Testing or Fatigue testing or Torture Testing. Stress tests examine how the system undergo intense loads and how it recovers when going back to normal usage. It is used for all types of software but it is particularly important for critical software. Also it emphasizes robustness, availability, and error handling under a heavy load rather than what is correct behavior under normal situations. It is a type of software testing for verifying stability and reliability of the system.
Objective of Stress Testing
- Testing beyond normal operational capacity, often to a breaking point, in order to observe the results.
- To confirm that the software does not crash in lacking computational resources like memory, disk space and network request.
- To ensure that the security of sensitive data after the failure are not compromised
- Stress testing can be used to discover hardware and data corruption issues.
- Stress testing will helps identify the security weaknesses that might sneak-in throughout constant peak load.
- It helps determine the software application’s data integrity throughout the extreme load, which implies that the data should be in a dependable state after a failure.
Why Stress Testing is needed?
Executing stress testing will enable developers and testers to be prepared for extreme situations in software. Stability of the software is also tested. If the system fails under stress, the company may face a tremendous loss in their revenue. Company will prepare the software to never fail its operation if working under heavy stress. It is important to perform stress testing as it helps the testing team to test the system in a situation of failure and to make sure that before crashing the system have saved the data or not and also to check whether any unexpected failures harmed the system security.
When Stress Testing should be performed?
Stress tests are significant for websites and apps before an online shopping festival or big events such as ticket selling for a famous concert with high demand of the people. So, it is very important to test regularly to the capacity of the system endurance. This also helps you to prepare for unexpected traffic spikes and gives more time and resources to fix any bottlenecks.
How to do Stress Testing?
Stress Testing process can be done in 5 major steps:
- Planning the Stress Test:Here you gather the system data, analyse the system, define the stress test goals
- Create Automation Scripts: Stress testing automation scripts are created, test data for the stress scenarios are generated.
- Script Execution:Run the Stress testing automation scripts and store the stress results.
- Results Analysis:Analyse the test results and identify bottlenecks.
- Tweaking and Optimization: Fine-tune the system, change configurations, optimize the code with goal meet the desired benchmark.
And lastly, again run the entire cycle to determine that the changes have produced the desired results.
Example of Stress Testing
- Education Board’s result website: – On the day of publishing main results, many students, users, and applicants will logins to the particular to check their grades.
- E-commerce website announces a festival sale: – Throughout the new product releases, sales, holidays, promotions, and festival offers, the e-commerce application tends to get many users in a very short time.
- Casually shut down and restart ports of a large network.
- The news website will crash or slow down when a major event happens.
Tools for Stress Testing
- Load runner
- Stress tester
- Neo load
Spike testing determine how well your system can cope with sudden traffic spikes. It is done by suddenly increasing and then decreasing the load generated by many users. It can be compared to a load or stress test, but modelled as a sudden burst of traffic.
Soak testing increases the number of concurrent users for an extended period of time and observe ability of software to tolerate extended periods of high traffic.
Also known as flood testing, it evaluate software’s ability to handle large volumes of data without breaking, slowing down, or losing any information.
It is a kind of Software Testing which is usually performed on initial software build to ascertain that the critical functionalities of the program are working fine. It is executed “before” any detailed functional or regression tests done on the software build. The purpose is to reject a badly broken application, so that the QA team does not waste time installing and testing the software application. Smoke testing is also known as “Build Verification Testing” or “Confidence Testing”. It is a part of Functional Testing.
Objective of Smoke Testing
Smoke testing means verifying the working of important features and ensuring there are no showstoppers in the build that is under test. It is a mini and rapid regression test of major functionality. It is a simple test that shows the product is ready for testing. This helps determine if the build is flawed as to make any further testing a waste of time and resources.
Why Smoke Testing is Performed?
Smoke testing is important because it’s an indicator of whether a build is ready for formal testing. Without it, major issues slip through the cracks and have the chance to stay within a build for longer—that’s why smoke testing lets developers squash bugs early on.
Smoke testing is done to ensure that the product is testable. It is performed in the beginning and detect the bugs in the basic features and send it to the development team so that the development team will have enough time to fix the bugs and the is done to make sure that the application is installed correctly.
When to do Smoke Testing?
A smoke test is performed at the beginning of the software development life cycle or SDLC process. Whenever the new build is installed, one round of smoke testing is performed because in the latest build blocker bug may be encountered. It is also called build verification testing. After all, there might be some change that might have broken a major feature (fixing the bug of or adding a new feature could have affected a major portion of the original software), or smoke testing can be done where the installation is happening.
How to do Smoke Testing?
A smoke test may be manual, automated, or a mix of the two. Usually, Smoke Testing is done manually to verify navigation is happening smoothly without any block to functionality. When the time is less and the new build is ready for deployment, automation can be used to smoke test. The process to implement an automated smoke test will vary depending on the application and the configuration of build tool used. Here are the basic steps for planning and running a smoke testing: –
- Prepare For Testing – After completing the build successfully, and before beginning to test it, setup tasks are performed. This might include copying files to the appropriate places, setting up database tables, installing licenses, or starting a server.
- Get Your Test Files –The next step is to gather the files required for the smoke test. Different smoke testing software has different command lines to get those files that need to be tested to the local drive.
- Script Writing – Single script for smoke testing will more flexibility. Also, the build script should be kept static at this time. One’s smoke test should run from their own build tool. Once it has run, the report should be saved with the rest of the build files. If something fails, it needs to be reported to the developers (along with an output of the script).
- Clean-Up – After the smoke test, need to clean up. This might include stopping a server, deleting files, or emptying database tables stop the server, and do certain other things. This could also be done before the initial setup step to ensure that the environment is clean for tests to start.
Example of Smoke Testing
For an eCommerce site the core working of this site should be login, specific search, add an item into the cart, add an item into the favourite, payment options, etc. Here we are testing function to place an order. After testing, the tester has to be sure and confident about the functioning of the function of the application.
Steps of the workflow as follows: –
- Click on item
- Description page should be open.
- Click on Add to Cart
- The cart should be open
- Click on Buy Now
- Payment options should be displayed. Choose one of them.
- Order placed
If this function is working correctly, then the test is pass and tester will test the next function of the same application.
Tools for Smoke Testing
Sanity testing refers to a subset of regression testing. It is a functional testing type. The sanity testing ensures that the changes made to the code do not adversely affect the system’s performance. After receiving software build, sanity testing is conducted to ensure that the changes made to the code are working correctly.
Objective of Sanity Testing
The objective is to determine that developer has applied sanity while producing the software. This type of testing is done on mature builds that have gone through multiple regression process and going to be released. Sanity test is done by tester
Why Sanity Testing is required?
Sanity test is done to verify existing functionalities are not changed while introducing new changes. It is an investment of time and effort. Sanity test helps to decide whether further testing can be carried forward or not
When to Perform Sanity Testing?
Sanity testing is done just before the deployment of software product. Software testing is done on builds received after bug fixing and also after performing minor change in the code.
How to Perform Sanity Testing?
Sanity testing is sometimes performed in an exploratory approach. To perform sanity testing there are two levels of test planning, high level sanity test planning and detailed sanity test planning.
Example of Sanity Testing?
You want to offer a 15% discount on New year for members having a premium membership on your online shopping application
- The sanity testing will verify the following:
- Premium members get the discount on the 1st
- members who do not have premium membership won’t get the discount on the 1st
- No member receives that discount offer before and after the 1st
- Other discount coupons continue to work.
- Premium members, who opt for other deals, do not get the new year discount.
Available Tools for Sanity Testing
Sanity testing is unplanned and totally manual so there are no special tools to perform it.
Security testing is a non-functional testing process that uncovers vulnerabilities of the system and prevents malicious attacks from intruders. It ensures that there is no threats or risks to the software system and application that can cause a loss.
Types of Security Testing
According to the Open-Source Security Testing techniques, different types of security testing are as follows:
- Vulnerability Scanning
- Security Scanning
- Penetration testing
- Risk Assessment
- Security Auditing
- Ethical hacking
- Posture Assessment
Objective of Security Testing
Main objective is to identify threats and security risks in the system. Principles of software testing are confidentiality, integrity, authentication, authorization, availability and non-repudiation. To identify threats and security risks in the system.
Why Security Testing is required?
It helps to avoid loss of customer’s trust, unexpected breakdown, revenue impact due to disturbance to your online system, costs required for repairing websites against future attacks.
When to do Security Testing?
Security testing is used to verify that there are secure interfaces between internal systems. And internal threats cannot be used to escalate privileges. This leads organization closer to a zero-trust security model.
How to perform Security Testing?
It is necessary to involve security testing in the software development life cycle in the earlier phases.
|SDLC Phases||Security Processes|
|Requirements||Check for abuse/misuse incidents and do a security analysis.|
|Design||For designing, security risk analysis is done. Developing a test plan that includes security tests|
|Coding and Unit Testing||Static and Dynamic Testing and Security White Box Testing|
|Integration Testing||Black Box Testing|
|System Testing||Black Box Testing and Vulnerability scanning|
|Implementation||Penetration Testing and Vulnerability Scanning|
|Support||Analyse the Impact of Patches|
Example of Security Testing
Sample test scenarios to give you an idea of security test cases-
- A password should be in an encrypted format
- Invalid users should not be allowed to access the application or system
- Check cookies and session time for application
- The browser back button should not work on financial sites
Free and Paid tools for Security Testing
- Acunetix – open source
- Intruder – it is paid online vulnerability scanner
- W3af- open-source Web Application Attack and Audit Framework
- Owasp- open source, Open Web Application Security Project
Smoke Testing Vs Sanity Testing
|Smoke Testing is performed to ascertain that the critical functionalities of the program are working fine.||Sanity testing is done at random to verify that each functionality is working as expected.|
|This testing is performed by the developers or testers||Sanity testing in software testing is usually performed by testers|
|Smoke testing exercises the entire system from end to end.||Sanity testing exercises only the particular component of the entire system.|
|The objective of the testing is to verify the “stability” of the system.||The objective of the testing is to verify the “rationality” of the system.|
|Smoke testing is usually documented and scripted.||Sanity testing is not documented and is unscripted.|
|Smoke testing can be either manual or automated.||Sanity testing can be done without test cases or scripts.|
|Smoke Testing is first performed on the initial build.||Sanity Testing is performed on stable build or for the new features in the software|
|It is a well elaborate and planned testing.||This is not a planned test and is done only when there is a shortage of time.|
|This is a wide and deep testing.||This is a wide and shallow testing.|
|Smoke testing is a subset of Acceptance Testing.||Sanity testing is a subset of Regression Testing.|
Stress Testing Vs Load Testing
|Load tests help you understand how a system behaves under an expected load||Stress tests help you understand the upper limits of a system’s capacity using a load beyond the expected maximum.|
|The attributes which are checked in a load test are peak performance, server quantity and response time.||This kind of testing checks stability response time, etc.|
|Load Testing is performed to test the “performance” of the system or software application under extreme load.||Stress Testing is performed to test the “robustness” and “ stability” of the system or software application under extreme load.|
|In load testing load limit is a threshold of a break.||In stress testing load limit is above the threshold of a break.|
|Large number of users.||Large number of users and large number of data|
|Load testing is performed to find out the upper limit of the system or application.||Stress testing is performed to find the behavior of the system under pressure.|
|The purpose of load testing is to generate more traffic for web application.||The purpose of stress testing is to prevent server crashes under sudden, high loads for an extended period.|
|The performance of the software is tested under multiple number of users.||The performance is tested under varying data amounts.|
|Load testing is used to test web-based and client server types of application.||Stress testing tests suddenly increased traffic to the application.|
|WebLOAD, LoadView, LoadRunner, SmartMeter.io, and LoadUI NG Pro are some of the load testing tools which help us to perform load testing on the applications.||LoadRunner, JMeter, NeoLoad are some of the stress testing tools which help us to perform stress testing on the applications.|
Advantages and Disadvantages of Performance Testing
|Measures the speed, accuracy and stability of software||Testing must be conducted out of hours during non-peak periods|
|Enhances client satisfaction by client retention measurements||Costly as it is a tedious process and carried out on multiple devices across multiple locations.|
|Early problem identification||Difficult to diagnose issues due to less instrumentation available to turn on in live|
|Improve optimization and load capability||It is high stake task|
|Supports project managers in the detection of error prior to publication||Performance issues may be causing problems to real users as code is already live|
Available Performance Testing Tools
- Apache JMeter
I hope you got detailed knowledge about why, when and how to do Load Testing, Stress Testing, Smoke Testing Security Testing and Sanity Testing. You can provide your feedback if you have any doubts in the comment box.