Skip to content

Creating a JMeter script compatible with PerfShop

This guide explains how to create a JMeter scenario from the JMeter GUI on your workstation, then upload it into PerfShop with Prometheus monitoring properly configured.

It incorporates the lessons learned during the development of PerfShop's bundled scenarios (JMeter 5.5 + XStream compatibility, XML pitfalls, plugin-free JSON extraction, Grafana label routing).


Prerequisites

  • JMeter 5.5 installed locally (download)
  • Prometheus Listener plugin installed in your local JMeter (optional for editing, required for PerfShop runs)
  • Access to the PerfShop JMeter UI (http://localhost:3005 or public URL)

Step 1 — Test plan structure

For metrics to surface by transaction name in the Grafana dashboard, each business step must be wrapped in a TransactionController with parent=true:

Test Plan
└── Thread Group
    ├── HTTP Request Defaults
    ├── CSV Data Set Config          ← user accounts
    ├── Cookie Manager               ← handles JSESSIONID automatically
    ├── Header Manager               ← global Content-Type / Accept
    ├── TransactionController [T01_Homepage]  ← parent=true
    │   ├── GET /api/products
    │   └── GET /api/products/categories
    ├── GaussianRandomTimer          ← think time between transactions
    ├── TransactionController [T02_SearchProduct]
    │   └── GET /api/products/search?q=...
    │       └── JSR223PostProcessor  ← JSON extraction (see §4)
    ├── ... (other transactions)
    └── Prometheus Listener          ← child of the Thread Group

TransactionController parent=true — key to Grafana label routing

With parent=true, the TransactionController generates a single aggregated sample whose label is the transaction name (e.g. T01_Homepage). This label is what the Prometheus plugin uses to break down metrics in Grafana. Without parent=true, every inner HTTP sampler surfaces individually — the dashboard becomes unreadable with many requests.

1.2 Naming transactions for Grafana

The TransactionController name becomes the Prometheus label. Adopt a convention:

T01_Homepage
T02_SearchProduct
T03_AddToCart
T04_Login
T05_PlaceOrder
...

The Grafana dashboard automatically filters out technical samplers via:

label!~"null|Pr[ée]parer.*|BeanShell.*|JSR223.*|Setup.*|Teardown.*|DEBUG.*|__.*"

Avoid these prefixes for your business transactions.

1.3 Configure target variables

Never hardcode the host and port. Use the JMeter properties injected by PerfShop:

In HTTP Request Defaults:

Field Value to enter
Server Name or IP ${__P(target_host,perfshop-app)}
Port Number ${__P(target_port,8080)}
Protocol ${__P(target_protocol,http)}

Step 2 — JMeter 5.5 + XStream compatibility (XML pitfalls)

Three critical rules for JMeter 5.5 compatibility

These rules stem from XStream errors encountered in production. Ignoring them causes NumberFormatException or NullPointerException when loading the JMX.

2.1 ThreadGroup.duration must be a stringProp

Problem: JMeter 5.5 uses XStream to deserialise JMX files. If ThreadGroup.duration is declared as a <longProp>, XStream tries to parse the value as a long before evaluating JMeter functions — causing:

Caused by: java.lang.NumberFormatException: For input string: "${__P(duration,300)}"

Rule: Always declare ThreadGroup.duration as <stringProp>, never <longProp>:

<!-- ✅ Correct -->
<stringProp name="ThreadGroup.duration">${__P(duration,300)}</stringProp>

<!-- ❌ Causes NumberFormatException with ${__P(...)} variables -->
<longProp name="ThreadGroup.duration">${__P(duration,300)}</longProp>

2.2 stringProp elements inside collectionProp must have a name attribute

Problem: In ResponseAssertion, HTTP codes are stored in a collectionProp. Every <stringProp> child must have a name attribute equal to the Java hashCode of its value, otherwise XStream throws:

Caused by: java.lang.NullPointerException
    at ConversionHelp.getUpgradePropertyName(ConversionHelp.java:253)

Rule: Use the Java hashCode of the value as the name attribute:

<!-- ✅ Correct -->
<collectionProp name="Asserion.test_strings">
    <stringProp name="49586">200</stringProp>
</collectionProp>

<!-- ❌ Causes NullPointerException -->
<collectionProp name="Asserion.test_strings">
    <stringProp>200</stringProp>
</collectionProp>

Java String.hashCode() values for common HTTP codes:

HTTP code name to use
200 49586
201 49587
400 51508
401 51509
403 51511
404 51512
409 51517
500 52469
503 52472

Step 3 — Plugin-free JSON extraction (JSR223PostProcessor)

JSONPathExtractor not available in the PerfShop image

JSONPathExtractor is provided by the bzm-json plugin (JMeter Plugins Manager). This plugin is not installed in the justb4/jmeter:5.5 image used by PerfShop. Using it causes:

Caused by: com.thoughtworks.xstream.mapper.CannotResolveClassException: JSONPathExtractor

Solution: Use a JSR223PostProcessor with native Groovy. Groovy and JsonSlurper are bundled in JMeter 5.5 with no plugin required.

3.1 JSON extraction pattern

<JSR223PostProcessor guiclass="TestBeanGUI" testclass="JSR223PostProcessor"
    testname="Extract securityToken" enabled="true">
  <stringProp name="scriptLanguage">groovy</stringProp>
  <stringProp name="parameters"></stringProp>
  <stringProp name="filename"></stringProp>
  <stringProp name="cacheKey">true</stringProp>
  <stringProp name="script">
try {
    def json = new groovy.json.JsonSlurper().parseText(prev.getResponseDataAsString())
    def val = json.securityToken
    vars.put("securityToken", val != null ? val.toString() : "NO_TOKEN")
} catch(Exception e) {
    vars.put("securityToken", "NO_TOKEN")
}
  </stringProp>
</JSR223PostProcessor>
<hashTree/>

3.2 Common JSON path equivalents

JSONPath equivalent Groovy access
$.securityToken json.securityToken
$.content[0].id json.content[0].id
$.orderNumber json.orderNumber
$.totalPages json.totalPages
$.id json.id

3.3 In the JMeter GUI

To create this post-processor graphically: right-click on the sampler → Add → Post Processors → JSR223 Post Processor


Step 4 — Add the Prometheus Listener

4.1 Position in the tree

The PrometheusListener must be a direct child of the Thread Group:

Thread Group
└── Prometheus Listener  ← ✅ collects all ThreadGroup metrics

And not:

Test Plan
└── Prometheus Listener  ← ❌ outside ThreadGroup → silent metrics

4.2 Configuration in the JMX

<com.github.johrstrom.listener.PrometheusListener
    guiclass="com.github.johrstrom.listener.gui.PrometheusListenerGui"
    testclass="com.github.johrstrom.listener.PrometheusListener"
    testname="Prometheus Metrics Exporter" enabled="true">
  <stringProp name="prometheus.port">${__P(prometheus.port,9270)}</stringProp>
  <stringProp name="prometheus.save.threads">true</stringProp>
  <stringProp name="prometheus.save.jvm">false</stringProp>
  <collectionProp name="prometheus.collector_definitions">
    <!-- jmeter_transactions_total (COUNTER) -->
    <elementProp name="" elementType="com.github.johrstrom.listener.ListenerCollectorConfig">
      <stringProp name="collector.metric_name">jmeter_transactions_total</stringProp>
      <stringProp name="collector.type">COUNTER</stringProp>
      <collectionProp name="collector.labels">
        <stringProp name="102727412">label</stringProp>
        <stringProp name="3059181">code</stringProp>
      </collectionProp>
      <stringProp name="listener.collector.listen_to">samples</stringProp>
      <stringProp name="listener.collector.measuring">CountTotal</stringProp>
    </elementProp>
    <!-- jmeter_response_time (SUMMARY P50/P95/P99) -->
    <elementProp name="" elementType="com.github.johrstrom.listener.ListenerCollectorConfig">
      <stringProp name="collector.metric_name">jmeter_response_time</stringProp>
      <stringProp name="collector.type">SUMMARY</stringProp>
      <collectionProp name="collector.labels">
        <stringProp name="102727412">label</stringProp>
      </collectionProp>
      <stringProp name="collector.quantiles_or_buckets">0.5,0.05|0.95,0.01|0.99,0.001</stringProp>
      <stringProp name="listener.collector.listen_to">samples</stringProp>
      <stringProp name="listener.collector.measuring">ResponseTime</stringProp>
    </elementProp>
    <!-- jmeter_success_ratio -->
    <elementProp name="" elementType="com.github.johrstrom.listener.ListenerCollectorConfig">
      <stringProp name="collector.metric_name">jmeter_success_ratio</stringProp>
      <stringProp name="collector.type">SUCCESS_RATIO</stringProp>
      <collectionProp name="collector.labels">
        <stringProp name="102727412">label</stringProp>
      </collectionProp>
      <stringProp name="listener.collector.listen_to">samples</stringProp>
      <stringProp name="listener.collector.measuring">SuccessRatio</stringProp>
    </elementProp>
  </collectionProp>
</com.github.johrstrom.listener.PrometheusListener>
<hashTree/>

Step 5 — CSV Data Set and variables

For multi-user scenarios, use a CSVDataSet with shareMode.thread (one distinct account per thread):

<CSVDataSet guiclass="TestBeanGUI" testclass="CSVDataSet"
    testname="User accounts" enabled="true">
  <stringProp name="filename">/scripts/users.csv</stringProp>
  <stringProp name="fileEncoding">UTF-8</stringProp>
  <stringProp name="variableNames">userEmail,userPassword,userName</stringProp>
  <stringProp name="delimiter">,</stringProp>
  <boolProp name="ignoreFirstLine">true</boolProp>
  <boolProp name="recycle">true</boolProp>
  <boolProp name="stopThread">false</boolProp>
  <stringProp name="shareMode">shareMode.thread</stringProp>
</CSVDataSet>
<hashTree/>

The users.csv file is mounted inside the container at /scripts/users.csv.


Step 6 — Upload and run in PerfShop

  1. Save the .jmx: File → Save Test Plan As
  2. Open PerfShop JMeter UI → 🗂️ Manage
  3. Choose the target folder → 📤 → select the .jmx
  4. Launch with ▶ Start run

Metrics appear in the perfshop-jmeter-live Grafana dashboard after ~10 seconds, broken down by TransactionController name.


Summary — Pre-upload checklist

Check point ✅ Expected
target_host ${__P(target_host,perfshop-app)}
target_port ${__P(target_port,8080)} — never 9080
ThreadGroup.duration <stringProp> — never <longProp> with ${__P(...)}
stringProp in collectionProp name attribute with Java hashCode of the value
JSON extraction JSR223PostProcessor Groovy — never JSONPathExtractor (plugin not installed)
TransactionController parent=true on each business step
PrometheusListener Child of the Thread Group, not the Test Plan
Prometheus port ${__P(prometheus.port,9270)}
Think time GaussianRandomTimer between each transaction
No hardcoded URL Neither localhost, nor a fixed IP, nor a public hostname

Step 7 — Step-by-step guide using JMeter GUI

This guide shows how to build a complete business journey scenario from the JMeter graphical interface, without editing XML by hand.

7.1 Create the test plan

  1. Launch JMeter: jmeter.bat (Windows) or jmeter (Linux/Mac)
  2. File → New for a blank plan
  3. Click Test Plan → rename: PerfShop — My scenario
  4. In the bottom panel, uncheck Functional Test Mode

7.2 Add a Thread Group

Right-click Test Plan → Add → Threads (Users) → Thread Group

Parameter Recommended value
Number of Threads ${__P(users,5)}
Ramp-up period ${__P(ramp_time,30)}
Duration ${__P(duration,300)}
Loop Count -1 (infinite with scheduler)
Scheduler ✅ Checked
On Sample Error Continue

Never hardcode users/duration

Always use ${__P(users,5)} — this allows the PerfShop JMeter UI to control the number of vUsers and duration without modifying the JMX file.

7.3 Add configuration elements

Right-click Thread Group → Add:

HTTP Request Defaults (Config Element → HTTP Request Defaults)

Field Value
Server Name ${__P(target_host,perfshop-app)}
Port ${__P(target_port,8080)}
Protocol ${__P(target_protocol,http)}
Encoding UTF-8

HTTP Header Manager (Config Element → HTTP Header Manager)

Add two headers:

Name Value
Content-Type application/json
Accept application/json

HTTP Cookie Manager (Config Element → HTTP Cookie Manager)

  • Clear cookies each iteration: ✅

CSV Data Set Config (Config Element → CSV Data Set Config)

Field Value
Filename /scripts/users.csv
Variable Names userEmail,userPassword,userName
Delimiter ,
Ignore first line
Recycle on EOF
Share Mode shareMode.all

shareMode.all — prevents account collisions

With shareMode.thread, each thread advances independently and multiple threads may land on the same account simultaneously. With shareMode.all, all threads share a global cursor that advances sequentially — each thread receives a different account per iteration. This is essential to avoid the "Duplicate order detected" error from the PerfShop backend (3-second anti-duplicate cooldown per user+cart).

7.4 Add a transaction

For each business step (T01, T02...):

  1. Right-click Thread Group → Add → Logic Controller → Transaction Controller
  2. Name it: T01_Homepage
  3. Check Generate parent sample: ✅ (this is the parent=true required for Grafana)
  4. Right-click the transaction → Add → Sampler → HTTP Request
  5. Configure the request: method, path, body

7.5 Add think time (GaussianRandomTimer)

Right-click Thread Group → Add → Timer → Gaussian Random Timer

Field Value
Constant Delay 2000 (ms)
Deviation 600 (ms)

Place this timer after each TransactionController (at Thread Group level, not inside it).

7.6 Add JSON extraction (JSR223PostProcessor)

Right-click the HTTP sampler → Add → Post Processors → JSR223 Post Processor

  • Script Language: groovy
  • Cache compiled script if possible: ✅
  • Script:
try {
    def json = new groovy.json.JsonSlurper().parseText(prev.getResponseDataAsString())
    def val = json.securityToken
    vars.put("securityToken", val != null ? val.toString() : "NO_TOKEN")
} catch(Exception e) {
    vars.put("securityToken", "NO_TOKEN")
}

7.7 Add a response assertion

Right-click the HTTP sampler → Add → Assertions → Response Assertion

  • Field to Test: Response Code
  • Pattern Matching Rules: Equals
  • Patterns to test: 200

To check 200 or 201 (OR): - Add 200 and 201 as patterns - Pattern Matching Rules: OR (check "OR")

7.8 Add a Groovy assertion (JSR223Assertion)

For complex checks (e.g. canCheckout = true):

Right-click the sampler → Add → Assertions → JSR223 Assertion

try {
    def json = new groovy.json.JsonSlurper().parseText(prev.getResponseDataAsString())
    if (json.canCheckout != true) {
        AssertionResult.setFailure(true)
        AssertionResult.setFailureMessage("canCheckout = false")
    }
} catch(Exception e) {
    AssertionResult.setFailure(true)
    AssertionResult.setFailureMessage("Invalid JSON: " + e.getMessage())
}

Never use JSONPathAssertion in PerfShop

Same rule as JSONPathExtractor: the bzm-json plugin is not installed. Always replace with a JSR223Assertion using Groovy.

7.9 Add a JSR223PreProcessor (before a sampler)

To prepare dynamic variables before a sampler executes:

Right-click the sampler → Add → Pre Processors → JSR223 Pre Processor

Example — randomly select a shipping method without __chooseRandom:

def methods = ["standard", "standard", "express", "premium"]
def r = new Random()
vars.put("shippingMethodToUse", methods[r.nextInt(methods.size())])

PreProcessor as child of a sampler: runs BEFORE the sampler

A JSR223PreProcessor positioned as a child of an HTTPSamplerProxy executes before that sampler. Variables it defines are available in the sampler's request body.

7.10 Add the Prometheus Listener

Right-click Thread Group → Add → Listener → jp@gc - Prometheus Listener

  • Port: ${__P(prometheus.port,9270)}
  • Save threads: ✅
  • Save JVM: ☐

Add three metrics (click +):

Metric Name Type Labels Measuring
jmeter_transactions_total COUNTER label, code CountTotal
jmeter_response_time SUMMARY label ResponseTime
jmeter_success_ratio SUCCESS_RATIO label SuccessRatio

For jmeter_response_time, add quantiles: 0.5,0.05|0.95,0.01|0.99,0.001

7.11 Save and test locally

  1. File → Save Test Plan As → name it my-scenario.jmx
  2. Run locally: Run → Start (or Ctrl+R)
  3. Verify in View Results Tree that all assertions pass
  4. Upload to PerfShop JMeter UI → 🗂️ Manage → 📤

Step 8 — Additional pitfalls discovered on PerfShop

These pitfalls come from debugging the parcours-metier-nominal.jmx scenario.

❌ Pitfall 6 — Groovy scripts on a single line with literal \n

Symptom:

startup failed:
Script166.groovy: 1: Unexpected character: '\' @ line 1, column 6.
   try {\n    def json = new groovy.json.JsonSlurper()...

Cause: When a JMX is edited manually and the Groovy script is placed on a single line with literal \n sequences (backslash + n), JMeter 5.5 / Groovy 3.0 does not interpret them as line breaks. The Groovy is syntactically invalid.

<!-- ❌ BAD — literal \n, single line -->
<stringProp name="script">try {\n    def json = ...\n} catch(Exception e) {\n}</stringProp>

<!-- ✅ CORRECT — real line breaks in XML -->
<stringProp name="script">
try {
    def json = new groovy.json.JsonSlurper().parseText(prev.getResponseDataAsString())
    vars.put("myVar", json.myField?.toString() ?: "default")
} catch(Exception e) {
    vars.put("myVar", "default")
}
</stringProp>

Rule: Groovy scripts in stringProp elements must always contain real XML line breaks.

❌ Pitfall 7 — __chooseRandom with trailing comma generates empty values

Symptom: Requests with 400/422, JSON body with empty fields ("street":"").

Cause: __chooseRandom(val1,val2,val3,) — the trailing comma is interpreted as an extra empty item. When JMeter picks the last item, the variable equals "" and the backend rejects the JSON.

# ❌ BAD — trailing comma
${__chooseRandom(standard,express,premium,)}

# ✅ CORRECT — no trailing comma (but still not recommended, see below)
${__chooseRandom(standard,express,premium)}

# ✅ BEST — JSR223PreProcessor with Groovy array
def methods = ["standard", "express", "premium"]
vars.put("method", methods[new Random().nextInt(methods.size())])

Rule: Never use __chooseRandom in PerfShop — always use a JSR223PreProcessor with a Groovy array.

❌ Pitfall 8 — && in inline JMeter functions causes XmlPullParserException

Symptom:

Caused by: org.xmlpull.v1.XmlPullParserException: entity reference names can not start with character '&'

Cause: The & character in an XML attribute must be escaped as &amp;. Inside a stringProp, && (Groovy logical AND) is invalid XML. The XStream parser (XmlPullParser) rejects it.

<!-- ❌ BAD — && invalid in XML stringProp -->
<stringProp name="Argument.value">{"id":${__groovy(vars.get('a') != null && !vars.get('a').equals('0') ? vars.get('a') : '1',)}}</stringProp>

<!-- ✅ CORRECT — separate JSR223PreProcessor BEFORE the sampler -->
// In a JSR223PreProcessor child of the sampler:
def a = vars.get("a")
vars.put("idToUse", (a != null && !a.equals("0")) ? a : "1")

Rule: All conditional logic must go in a separate JSR223PreProcessor. Never put &&, ||, or <> in a stringProp without XML escaping.

❌ Pitfall 9 — Backend anti-duplicate protection (HTTP 500)

Symptom: POST /api/orders returns 500 with "Commande en double détectée" under load.

Cause: The PerfShop backend implements an anti-double-click protection (anomaly A5). In nominal mode (chaos level 0), if the same user creates two orders with the same cart contents within a 3-second window, the second is rejected with a RuntimeException → Spring returns HTTP 500.

Under load with multiple threads sharing the same account (e.g. user1@perfshop.com), this window is easily exceeded.

JMX fix: Use shareMode.all in the CSV Data Set so that threads share accounts sequentially and never have the same active user simultaneously.

Assertion fix: Accept the 500 duplicate as a conditional success via JSR223Assertion:

def code = prev.getResponseCode()
if (code == "500") {
    def body = prev.getResponseDataAsString()
    if (body.contains("doublon") || body.contains("duplicate")) {
        // Anti-duplicate protection triggered — normal under load
        return
    }
}
def json = new groovy.json.JsonSlurper().parseText(prev.getResponseDataAsString())
if (json.success != true) {
    AssertionResult.setFailure(true)
    AssertionResult.setFailureMessage("success=false : " + prev.getResponseDataAsString().take(200))
}

❌ Pitfall 10 — Empty orderId in URL causes URISyntaxException

Symptom:

Non HTTP response code: java.net.URISyntaxException

Cause: If T08 (order creation) fails, ${orderId} remains empty. The URL /api/orders//details is syntactically invalid for java.net.URI.

Fix: Add a JSR223PreProcessor before the GET /api/orders/${orderIdToFetch}/details sampler that sets a fallback of "0" if orderId is missing:

def oid = vars.get("orderId")
if (oid == null || oid.equals("") || oid.equals("0")) {
    vars.put("orderIdToFetch", "0")
} else {
    vars.put("orderIdToFetch", oid)
}

Use ${orderIdToFetch} in the path instead of ${orderId}.


Complete summary — Definitive JMX rules for PerfShop

Rule Description
shareMode.all CSV Data Set — sequential account distribution across threads
✅ Multi-line Groovy scripts Real line breaks in stringProp script elements
JSR223PreProcessor for logic Never use &&, ||, __groovy() inline in stringProp
JSR223PostProcessor for extraction Never use JSONPathExtractor (plugin not installed)
JSR223Assertion for JSON assertions Never use JSONPathAssertion (plugin not installed)
JSR223PreProcessor for random data Never use __chooseRandom with trailing comma
✅ Fallback on critical variables Check orderId, foundProductId, foundProductPrice before use in URLs
TransactionController parent=true Required for Grafana metric ventilation
${__P(...)} for all configurable params Never hardcode users, duration, target_host
<stringProp> for ThreadGroup.duration Never <longProp> with ${__P(...)}
✅ HashCode name attribute on stringProp in collectionProp Avoids XStream NPE
✅ PrometheusListener child of ThreadGroup Not of TestPlan