Performance Testing & Engineering trends of 2020

In this age of Digital Transformation, application performance has become the most important part of business. It is not just about page load or page speed, rather drives the business revenue by ensuring the software can process transactions in a responsive manner for a greater customer experience. Therefore, performance testing/engineering is emerging as a specialization in software testing space and software testers are encouraged to adopt multi-layered testing approach instead of traditional performance testing. In this article, we will talk about some of the best practices that will help testers to uncover different aspects of performance testing & engineering, at a high level.

Let’s start with discussing about Test Strategy around performance testing and will also talk about some of the recent trends in Performance Engineering in the Industry in later part of this article.

Building a Complete Test Strategy

Performance Testing & Engineering requires deep technical knowledge about application & technology which is followed by detailed planning and preparations before execution of test. BA/Product owner, Business Owner, Application developer, Production Monitoring Team are the several participants in planning phase. During planning & strategy prep, we must consider including shift-left and shift-right load testing approach.

a) Shift-Left Load testing

Shift-Left testing means doing testing in early or every stage of development. It saves time and improves overall quality of application. It doesn’t require to wait until the end of development process to do performance testing for the entire application.

b) Shift-Right Load testing

Shift-Right testing has big impact on overall quality of application. Applying test practices to production means considering real-world users and their experiences into the development process.

Right workload model & plan

Right Workload model is always a challenge for Performance Test engineer. Test engineer needs to carefully define think time, pacing, Critical transaction and users’ Load.  An inaccurate workload misguides the server optimization and sometimes it might also delay project deployment.

Emphasis on baseline tests or one-user load test

One-user load test is important for application baselining. It saves time and helps the tester to find defects in full load test. It isolates performance defects in early round of testing (Never forget to consider one-user load test in your plan).

Identifying the performance bottleneck

Identifying bottleneck is always a challenge for the tester. It requires deep technical knowledge and expertise. There are few techniques generally used by performance engineers:

1. Trend analysis

It is a technique that is commonly used to find out the behavior of the data related to the frequency of the performance issues in order to find out the bottleneck. Performance bottleneck can be of various types such as consistent, fluctuating, increase and decrease. This analysis can be done on both client-side and server-side metrices.

2. Correlation

It evaluates the relationship between different performance metrices in order to find out the bottlenecks. It is like comparing user’s response time graph with CPU graph during the test to uncover the performance issues or to do the analysis for the subsequent steps.

3. Comparison

As the name suggests, here we compare the performance metrices of the baseline test with unacceptable performance test metrices. This helps in finding out the reasons of the bottlenecks.

4. Elimination

Using elimination technique we remove certain components during the test to identify real bottlenecks e.g.  removing web server and doing load test directly for the application server.

Creating report

Effective reporting of test results is one of the important & key task for a performance tester. Final performance test report provides detailed analysis and recommendations to the project team and also the business. All the stakeholders make their critical Go/No-Go decisions based on this report. Test report must include:

1. Findings/observations
2. Detailed description of performance test results
3. Recommendations
4. Defect details
5. Sign offs

Conclusion

Performance testing requires deep technical skills for different technologies. The success of performance test depends upon how we plan and execute. I hope this article helps to outline few critical aspects of successful performance test planning & execution. Performance test engineers need to consider the above tricks and techniques in order to achieve significant performance enhancements of the application under test (AUT).

I wanted to keep this article as a starting point to give you a summary around the latest performance engineering trends but keep checking this space for few more articles on performance testing & engineering shortly.