Result Mismatch between Jmeter and Webload in Performance testing

Question

i'm new into performance testing. While doing performance testing of a website I found the results obtained by Jmeter were not matching with the results from Webload. So, i tested a simple website lifechanger.org through both the tools, with 1 Virtual user but still I got different different values. Can anyone has any help on this. Please read below values obtained through both the tools.

URL: Lifechanger.org

Only Throughput matched in this scenario. Any help will be grateful. Thanks in advance.

Answer 1

You should not be comparing 2 load testing tools against each other, you should be comparing them to the real browser and configure both to match real browser's behavior, otherwise your test will not make a lot of sense.

Things to consider:

• You need to make sure that you set the load testing tool to retrieve all embedded resources from the page (images, scripts, styles, fonts, etc.)
• You need to make sure it's being done in parallel (as this is what real browsers do)
• You need to make sure it's being done only once (or according to Cache-Control headers), in other words implement browser's cache.
• You need to accept/send back HTTP Cookies
• You need to send appropriate HTTP Headers like User-Agent or Accept-Encoding
• You need to properly simulate AJAX calls, like embedded resources they're being executed in parallel

Ideally well-behaved load testing tool should give you the same timings as browser developer tools on its Network tab.

More information:

• How to make JMeter behave more like a real browser
• WebLoad Scripting Guide

Answer 2

As throughput and hits/sec were matching and we were having different response time and latencies on Jmeter and webload, I did some research on the time difference on Latencies and response time. Please find below observations.

Latency in Jmeter : JMeter measures the latency from just before sending the request to just after the first response has been received. Thus the time includes all the processing needed to assemble the request as well as assembling the first part of the response, which in general will be longer than one byte. Protocol analyzers (such as Wireshark) measure the time when bytes are actually sent/received over the interface. The JMeter time should be closer to that which is experienced by a browser or other application client.

Latency in Webload: The time that elapsed since a request was sent until the Virtual Client received the first byte of data.

As Jmeter is working from perspective of application client where it’s also considering the time taken on post processing of DOM, that may be the reason we are having increased time shown up in Jmeter which is in seconds. This is the way Jmeter works on the other hand Webload is showing time without processing of data, the time it takes to receive data in DOM, which is always in millisec.

References: 1). https://jmeter.apache.org/usermanual/glossary.html 2). Webload Help Section in the app