Category Archives: JMeter

Debugging JMeter Plug-ins

When developing your own plugins, you’ll no doubt need to debug some sort of problem sooner or later. Writing log messages or even just printing directly to stdout will often give you the info you need, but eventually, you’ll want to hook up a debugger and step through your code. Fortunately, this is very easy and just needs the following lines added to the top of your start up scripts:

Linux/Mac – bin/jmeter.sh

JVM_ARGS="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000"

Windows – bin/jmeter.bat

set JVM_ARGS=-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000

Now when you launch JMeter using either of these scripts, the JVM will start up with the debugger enabled and listening on port 8000. You can then use your IDE of choice to debug your plug-in. In my case, I’ve used Eclipse. To set up the debugger connection, select Debug Configurations… from the Run menu and you should see a dialog similar to the following:

The important things to note here is that you want to add a new Remote Java Application debug configuration and the project should be set to your plug-in project. Optionally, you might also want to add this configuration as a favourite to your debug menu which can be done on Common tab.

When you launch this configuration, the Eclipse debugger will establish a connection to your running JMeter instance and you will then be able to set break points and step through your code as normal. As an added bonus, if you’ve also downloaded the JMeter source code, you’ll be able to attach it to the JMeter Jars in your project allowing to step through the JMeter code as well without having to build it yourself. This is a great way to learn how JMeter works internally which may help you better understand any problems you see with you plug-ins.

Faster JMeter If Controllers

When load testing to a large scale, it’s just as important for your test scripts to be optimized as it is for your servers. Your tests will often need to simulate thousands of users on a single physical box running JMeter and so you need to be running as efficiently as possible. If you are using the standard If Controller in your test with their default configuration, then it’s likely that you’re burning CPU cycles and resources just to check if two variables are the same.

By default, the If Controller will use JavaScript to evaluate the condition you’ve specified. This means that JMeter will create a brand new JavaScript execution context each time an If Controller is processed. This happens for every controller in every thread. That will give your garbage collector a good work out as well as your servers!

Fortunately, the JMeter team knew this could be a problem and so added the option evaluate the condition as a variable expression. This allows you to create a JMeter function to check your condition that returns either true or false. This is tens to hundreds of times faster.

Creating custom functions is even easier than creating a test component. You just need to add the class file to your plugin project and JMeter will automatically find it and do the rest. Here’s another article that goes into the specifics of JMeter functions a bit more and you can also read my original guide on creating custom components.

Common Checks

The most common checks I perform in my scripts are to test if a variable is equal to some value. Using a JavaScript condition, you would specify something like ${value} == 1. However, for my tests, I’ve created my own __eq() function for quickly comparing two values:

[code language=”java”]
package org.adesquared.jmeter.functions;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;

import org.apache.jmeter.engine.util.CompoundVariable;
import org.apache.jmeter.functions.AbstractFunction;
import org.apache.jmeter.functions.InvalidVariableException;
import org.apache.jmeter.samplers.SampleResult;
import org.apache.jmeter.samplers.Sampler;

public class EqFunction extends AbstractFunction {

private final static String NAME = "__eq";
private final static ArrayList<String> DESC = new ArrayList<String>(2);

private CompoundVariable a;
private CompoundVariable b;

static {
DESC.add("a");
DESC.add("b");
}

@Override
public String getReferenceKey() {
return NAME;
}

public List<String> getArgumentDesc() {
return DESC;
}

@Override
public String execute(SampleResult previousResult, Sampler currentSampler) throws InvalidVariableException {

String a = this.a.execute();
String b = this.b.execute();

return a.equals(b) ? "true" : "false";

}

@Override
public void setParameters(Collection<CompoundVariable> parameters) throws InvalidVariableException {

if (parameters.size() < 2) throw new InvalidVariableException("Not enough parameters for " + NAME);

Iterator<CompoundVariable> it = parameters.iterator();
this.a = it.next();
this.b = it.next();

}

}
[/code]

Using that, I can enable the Interpret Condition as Variable Expression option in the If Controller and change the condition to ${__eq(${value},1)}. That will give me exactly the same functionality as before but without JMeter creating a new JavaScript context each time.

How much faster?

To test how much faster, I put together a small script that had a 100,000 iteration Loop Controller that contained an If Controller that contained a Test Action Sampler (a sampler that will do nothing in this case). I defined the variable value to have the value 1 in the test. You can download the test script from here.

When the test is executed using JavaScript to interpret the condition, it takes 78 seconds to complete the test. Changing the condition to a variable expression using my new __eq() function, the test took less than 2 seconds. Both tests ran using the server VM.

I know which one I’m going to continue to use in the future!

Bonus Not Equals Function

Of course I also often need to check if two values aren’t equal, for those, I use this function:

[code language=”java”]
package org.adesquared.jmeter.functions;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;

import org.apache.jmeter.engine.util.CompoundVariable;
import org.apache.jmeter.functions.AbstractFunction;
import org.apache.jmeter.functions.InvalidVariableException;
import org.apache.jmeter.samplers.SampleResult;
import org.apache.jmeter.samplers.Sampler;

public class NeqFunction extends AbstractFunction {

private final static String NAME = "__neq";
private final static ArrayList<String> DESC = new ArrayList<String>(2);

private CompoundVariable a;
private CompoundVariable b;

static {
DESC.add("a");
DESC.add("b");
}

@Override
public String getReferenceKey() {
return NAME;
}

public List<String> getArgumentDesc() {
return DESC;
}

@Override
public String execute(SampleResult previousResult, Sampler currentSampler) throws InvalidVariableException {

String a = this.a.execute();
String b = this.b.execute();

return a.equals(b) ? "false" : "true";

}

@Override
public void setParameters(Collection<CompoundVariable> parameters) throws InvalidVariableException {

if (parameters.size() < 2) throw new InvalidVariableException("Not enough parameters for " + NAME);

Iterator<CompoundVariable> it = parameters.iterator();
this.a = it.next();
this.b = it.next();

}

}
[/code]

Using JMeter’s Table Editor

Following on from my previous post about creating custom JMeter components, I thought it worth taking a look at using the test bean table editor as it’s a good way of editing lists for your own components. Unfortunately, it doesn’t have much documentation and only actually works due to type erasure!

I’ve put together another sample that uses the table editor to show how it works. This time, I’ve created a config element that resets variables on each loop iteration. This is different to the User Defined Variable components as that will only set the variables once.

Table Editor Class Overview

The table editor can be found in the org.apache.jmeter.testbeans.gui package along with a number of other additional property editors. It will allow you to edit properties that are set using a list of classes (e.g. a list of key/value pairs) without having to try to create a complex interface using the standard text boxes. An example of using a table is the User Defined Variables config element (although this doesn’t actually use the table editor that’s available to test beans).

To use the table editor, you must define a class the represents a row of data in the table. This class must have a public, zero argument constructor and get/set property functions for the columns you want to edit. In addition to this, the class must also extend the AbstractTestElement class. This is what caused me a lot of problems initially as it’s not mentioned in the docs and although it may appear to work if you don’t do this, you will run into problems such as breaking the JMeter GUI or not being able to save your tests.

Once you’ve extended that class, you must also make sure that you save any of your properties in the AbstractTestElement properties map otherwise they won’t be saved to the JMX file or passed to your components correctly when you run your test. In my table editor sample, I used the following class for my row data:

[code language=”java” firstline=”23″]
// A class to contain a variable name and value.
// This class *MUST* extend AbstractTestElement otherwise all sorts of random things will break.
public static class VariableSetting extends AbstractTestElement {

private static final long serialVersionUID = 5456773306165856817L;
private static final String VALUE = "VariableSetting.Value";

/*
* We use the getName()/setName() property from the super class.
*/

public void setValue(String value) {
// Our property values must be stored in the super class’s or they won’t be saved to the JMX file correctly.
setProperty(VALUE, value);
}

public String getValue() {
return getPropertyAsString(VALUE);
}

}

[/code]

My property functions in my component are:

[code language=”java” firstline=”47″]
// Our variable list property
private List<VariableSetting> settings;

public void setVariableSettings(List<VariableSetting> settings) {
this.settings = settings;
}

public List<VariableSetting> getVariableSettings() {
return this.settings;
}
[/code]

Configuring the table editor

The table editor is configured through property descriptor variables in your bean info class and so is really easy to set up. The only additional work you need to do is manually request the localised strings for your column headers from your resource file. Here’s my bean info file as an example:

[code language=”java”]
package org.adesquared.jmeter.config;

import java.beans.PropertyDescriptor;
import java.util.ArrayList;
import java.util.ResourceBundle;

import org.adesquared.jmeter.config.ResetVariablesConfig.VariableSetting;
import org.apache.jmeter.testbeans.BeanInfoSupport;
import org.apache.jmeter.testbeans.gui.TableEditor;

public class ResetVariablesConfigBeanInfo extends BeanInfoSupport {

private final static String VARIABLE_SETTINGS = "variableSettings";
private final static String HEADER_NAME = "header.name";
private final static String HEADER_VALUE = "header.value";

private final static ArrayList<VariableSetting> EMPTY_LIST = new ArrayList<VariableSetting>();

public ResetVariablesConfigBeanInfo() {

super(ResetVariablesConfig.class);

// Get the resource bundle for this component. We need to do this so that we can look up the table header localisations
ResourceBundle rb = (ResourceBundle) getBeanDescriptor().getValue(RESOURCE_BUNDLE);

PropertyDescriptor p;

p = property(VARIABLE_SETTINGS);
p.setValue(NOT_UNDEFINED, Boolean.TRUE);
p.setValue(DEFAULT, EMPTY_LIST);

// Set this property to be edited by the TableEditor
p.setPropertyEditorClass(TableEditor.class);
// Set the class that represents a row in the table
p.setValue(TableEditor.CLASSNAME, VariableSetting.class.getName());
// Set the properties for each column
p.setValue(TableEditor.OBJECT_PROPERTIES, new String[] {
"name",
"value"
});
// Set the table header display strings
// These must be read directly from the resource bundle if you want to localise them
p.setValue(TableEditor.HEADERS, new String[] {
rb.getString(HEADER_NAME),
rb.getString(HEADER_VALUE)
});

}
}
[/code]

And how the table editor shows up in the JMeter GUI:

And once again, you can see how easy it is to extend JMeter with your own components when you know how.

A final note on type erasure

As I mentioned earlier, the table editor only works due to type erasure and I wanted to look at it quickly as it’s something that catches quite a few people out. In Java, type erasure is the loss of generic type specification at runtime. A List<String> becomes a plain old List. This means that at runtime, it is possible to assign a List<String> reference to a List<HashMap<Integer, Boolean>> reference without any problems until you try to get a value from the list. Often, the compiler can spot these problems and will report an error, but as with anything, it’s possible to trick the compiler.

Here’s the code from JMeter that takes advantage of this:

[code language=”java” firstline=”285″ highlight=”299″]
/**
* Convert a collection of objects into JMeterProperty objects.
*
* @param coll Collection of any type of object
* @return Collection of JMeterProperty objects
*/
protected Collection&lt;JMeterProperty&gt; normalizeList(Collection&lt;?&gt; coll) {
if (coll.isEmpty()) {
@SuppressWarnings(&quot;unchecked&quot;) // empty collection
Collection&lt;JMeterProperty&gt; okColl = (Collection&lt;JMeterProperty&gt;) coll;
return okColl;
}
try {
@SuppressWarnings(&quot;unchecked&quot;) // empty collection
Collection&lt;JMeterProperty&gt; newColl = coll.getClass().newInstance();
for (Object item : coll) {
newColl.add(convertObject(item));
}
return newColl;
} catch (Exception e) {// should not happen
log.error(&quot;Cannot create copy of &quot;+coll.getClass().getName(),e);
return null;
}
}
[/code]

What this function is doing is accepting a collection of any type and converting it to a collection of JMeterProperty objects. However, on line 299, a new collection is created using the class of the collection that is passed in. This new collection is then assigned to a Collection<JMeterProperty> reference. When using the table editor, the collection passed into this function will be of whatever type you have used for your table row data (Collection<VariableSetting> in the case of my sample). The compiler has obviously picked up on creating and assigning an object this way as being unsafe which is why the @SuppressWarnings annotation has been added.

However, this isn’t a mistake or bug in the code, this is quite intentional and a clever way of constructing a new collection with the same implementation as the input collection. Collection<E> is just an interface and so you can’t construct a collection object directly. Instead, you must construct something like a HashSet or an ArrayList, objects that implement the Collection interface. The function above has decided not to make any assumption about which implementation the normalised collection should use and instead constructs a new collection using the input collection’s class.

So, if the input collection was of type ArrayList<String>, the collection that is returned will be ArrayList<JMeterProperty>, the same implementation but storing a different object type.