Software testing by Unify

Performance Testing Tools

In 2009, Amazon found that for every 100ms of latency, it cost them 1% of sales. And for a huge company such as Amazon, this translates to millions of pounds worth of sales and lost revenue. Google also found that 0.5 seconds in search page generation dropped traffic by 20%, which ultimately lost them revenue in the form of advertising reach and analytics.

To test the performance of a system, there are many components “under the bonnet” that need to be fit for purpose and functionally correct.

When performance testing a system, there are many considerations, some of which are listed below:

Software

The performance testing tool gives you control over your tests and the dashboard to see the metrics and results of your tests. It is vital to choose the correct tool, based on your requirements and capabilities within the team.

Skill

Some tooling can be difficult to learn. A tool like JMeter is popular within the industry due to its codeless nature and ease of use, but there may be skill within the test team which allow adoption of different tooling.

Load Resource

Your personal or work PC doesn’t have the resource to simulate acting as thousands of virtual users without completely exhausting the resources of the machine. It may be a requirement to have access to resources to be able to push a larger number of virtual users through your system. by using additional in-house solutions or cloud infrastructure like Amazon Web Services (AWS).

Environment

Performance testing is pointless if you don’t have access to a like-for-like environment of production to test against.
Testing against QA will not provide accurate results for live. You also can’t scale up from the difference in resource from QA, as it won’t yield accurate results if there is a bottleneck in the system.
Testing against production is dangerous and can be extremely costly if the entire site grinds to a halt.
You will need to have access to a 1:1 environment for production, with the exact same configuration in order to have accurate results.

There are many solutions that can provide the software and test design. These are the most commonly used:

JMeter

JMeter is an open source Java application, with the capability to design and execute performance tests. Its major limitation is that it’s only a piece of software, meaning you would have to provide the infrastructure to be able to run loads. You can still run small loads of virtual users as long as it stays within the resource pool of your device.
Codeless solution makes it easy to learn and execute. This solution is free.
Record and playback feature.
Can be found at https://jmeter.apache.org/

Taurus

Taurus is an open source test automation framework built with the intent to test performance of a system. It has the same drawbacks as JMeter, in that it doesn’t provide infrastructure and only runs on the machine you’re using.
Write tests with JSON or YAML.
This solution is free.
Can be found at https://gettaurus.org/

BlazeMeter

A cloud solution that can take Taurus tests or JMX files, BlazeMeter offers the ability to also record tests with its Chrome extension.
BlazeMeter also provides integration into Azure or Google Cloud to allow tests to be run on cloud systems, so you don’t need your own infrastructure to be able to run massive loads.
Very easy to configure tests to use different engines in different countries and change the virtual numbers of a test without having to create a new test for it.
Multiple pricing options depending on the load required.
Can be found at https://www.blazemeter.com/

Gatling

Gatling is a coded performance testing solution where tests are written in Scala and run on the machine – this is their free solution which is open source.
Gatling offers a paid management suite, which also offers the infrastructure to be able to load tests with either Azure or AWS.
Tests can be written in the free version to avoid costs and then executed in their management suite.
Free option and also paid management suite that includes infrastructure.
Can be found at https://gatling.io/

There are dozens more to choose from and we’re experts at help you find the right fit. Get in touch to find the right solution for you.

+44 (0)114 399 2820