What is performance testing ?
Performance engineering is a software engineering discipline that deals with the techniques, processes, and metrics to make our applications more performant. It is a technique by which we tune the code that we write by measuring against the nonfunctional requirements, such as performance under a certain amount of concurrent users or sessions. Usually performance testing involves simulating production-level workloads in a testing environment to understand how the application performs.
Why do we need to do performance testing ?
We live in the cloud era where we are enabled by platforms like Azure that have helped move the infrastructure costs of our workloads from Capital expenditure (need to procure servers and network devices upfront) to an Operating expenditure model (provisioning servers on demand) . We now can provision additional servers just-in-time, and we can scale out and scale down easily based on runtime metrics like CPU utilization and storage queue length.  In addition, we also know that carbon emissions are directly proportional to the servers we set up. From Sara Bergman’s articles – How can I calculate carbon emissons for my Azure frond applications and Measuring power consumption for my backend services , it is clear that the more infrastructure we setup the higher the carbon footprint.
How does it help in Carbon savings?
Performance testing helps optimize the infrastructure needed for running a workload. Consider a web application that connects to the database and is used concurrently by more than 1000 users within a geo-region. A typical infrastructure setup is as per this architecture Basic web application – Azure Reference Architectures | Microsoft Docs .
It has an Azure SQL database behind an App service. There are multiple pricing tiers you can consider to provision for this application, but we can use performance engineering techniques to derive the right tier. A high-level set of activities covering these techniques are:
- Identify the user load, latency acceptable and throughput required for the application (Non-functional requirements – NFR)
- Choose the lowest pricing tier for the Azure services that has specifications matching the NFRs.
- Run a simulated load test on key application scenarios and identify the headroom available in the infrastructure to take on additional load.
In the example graph below, the utilization of the database was calculated after the initial load tests. As you can see, the average utilization is hovering around 80% with a peak close to 97%. This does not allow any excess capacity to add additional workloads on the database. Running Azure Database tuning advisor helped identify many recommendations, including need to add indexes and to fix some long running queries . Incorporating the recommendations helped bring down the database utilization as you can see in the “After Tuning” graph. Now the same database at the same pricing tier can handle additional application querying.
Before Tuning | After Tuning |
Another Example of Tuning:
In another example, you can see that during the initial performance runs, an application is showing high CPU utilization and is also showing memory leaks (since memory graph is not reducing at the end of the load run). This is another scenario where we could have iteratively tried provisioning more infrastructure (higher RAM, premium pricing tier ) and the memory leak and CPU utilization problems could have been contained. This however means higher carbon footprint and infinitely increasing costs for the customer. Code written poorly is never identified and the infrastructure used is not optimal for the workload. Tuning techniques like asynchronous callback, caching and explicit garbage collection calls were implemented to fix the code issues . This helped bring down the CPU utilization and now it has enough excess capacity to handle more workloads. This is an example of a energy efficient application.
Before tuning | After Tuning |
In Summary
The art and science of performance engineering will help us:
- Build applications that are energy efficient, thus minimizing the amount of carbon emitted per unit of work.
- Run servers and machines at a high rate of utilization. Take full advantage of what you have already and minimize wasted cycles and resources. This technique makes the applications more hardware efficient
To read and learn more on how to make your applications energy efficient and hardware efficient , I highly recommend to go over the articles in Green Software Foundation
“(…) and explicit garbage collection calls were implemented to fix the code issues”
That’s interesting – why GC was not able to do its job and explicit calls were necessary? Could you shed some light on this?
For managed frameworks, garbage collection process would do. But we had objects created that were handling calls to unmanaged frameworks as well.