How can using Apache JMeter as a testing automation tool increase our confidence as testers? I’ve spent quite a lot of time in recent months working with and learning JMeter. While I’m still not nearly as adept at is at I’d like to be in the future, I have quite a few takeaways that have really changed my mindset on the tool and automation testing as a whole.

 JMeter is not only a performance testing tool

Upon first hearing about JMeter and dipping my toes into its open source waters, I was under the initial impression that it was a tool for load testing. This may have been due to our priorities at the time and the fact that we were investigating it for use with load testing, but for some reason my mind stuck to that point. As I’ve researched it and made adjustments to my test plans, I have come to realize that JMeter is capable of much more.

JMeter is first and foremost an automation tool. When we write test plans to run, whether they, in fact, be for load testing or stress testing, they are written to run automatically. That’s how the load is possible. But do we look at our results in a way that says “These test cases were actually run during my load tests, what do the results tell me about them?” Or do we primarily look at resource usage and other performance indicators. If you want, you can even write JMeter tests specifically for automation, rather than load. For instance, using REST APIs you are able to set up test plans in JMeter that will run throughout your workflows and testcases automatically. With “listeners” you are able to see the results, down to every variable and property, for every call. You can see how long they take, if they are completing successfully, if the responses are as expected, etc. This is a much deeper look into everything than simply clicking a “Delete” button in the UI and making sure it works. You can easily boost your confidence into your test cases when seeing every inch of the backend, and having control over the load, conditions, and scenario.

     What I learned this week

     For most of my time using JMeter so far, I have been using it through the GUI application. It’s great! I can see the results, modify on the fly, and have full control right there. But upon doing research I have realized that running it through the command line is much better for when you’re actually ready to perform your tests. You can set up your test plan’s logging by using the properties files as well as use commands and have flexibility for the test at runtime, rather than hardcoded. You can specify right then and there which logging to use, which properties to use, etc. You are also not limited based upon the GUI and the memory that it takes. The JMeter GUI uses up a lot of computer memory and can have an impact on your test performance.

You can achieve this by running the following command:

jmeter -n –t test.jmx -l testresults.jtl

jmeter is the prefix for any jmeter commands. -n tells the command to run in non gui mode. -t (testplan) selects the jmx file to run. -l (logfile) specifies an output file for logs.

Properties files were probably the biggest leap in my test plans this week. Previously most of my properties and variables were set up within the test plan, hard coded and changed in the GUI if needed. Upon switching my execution of JMeter tests to the non-GUI method, I had to make adjustments which not only made it possible to run from a command line, but also made my tests much more flexible over all. By using properties files and specifying them at runtime, we can set up such things as:

  • Environments
  • Logging
  • Variables
  • Thread Counts
  • Loops
This allows our tests to be flexible and changeable with each time we run them, without ever having to open jmeter.bat and modifying everything. JMeter comes with default properties files, but  I’ve found it useful to create a blank one, and fill it with properties that you’d like accessible by your entire test plan.  You can access these directly by using ${__P(“name”,”defaultName”)}. This will use defaultName (or whatever you’d like as a default value) if “name” is not found in the properties file.
I strongly encourage looking into properties files and seeing how you can keep your JMeter test plans flexible, run them through the command line, and think of them as an overall testing tool, rather than simply for load tests. The help they provide can surprise you!