Test Runner¶

perfshop-test-runner is the execution container where all Robot Framework and pytest tests actually run. It is a custom image built locally from the Dockerfile in test-runner/ — it is not pullable from a public registry.

Its sole role: receive Python (pytest) or Robot Framework scripts through two distinct channels (SSH for Squash Orchestrator, docker exec for Scripts UI), run them by driving Selenium, and produce artifacts (logs, reports) consumable by the other services in the stack.

Source of truth

This page is drawn from the test-runner/Dockerfile, the perfshop-test-runner block in the compose files, and test-runner/docker-entrypoint.sh.

Custom Dockerfile¶

Base image and system dependencies¶

FROM python:3.11-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
    openssh-server \
    git \
    && rm -rf /var/lib/apt/lists/*

Only three system packages:

Package	Usage
`python:3.11-slim` (base)	Python runtime and pip
`openssh-server`	Lets Squash Orchestrator connect via SSH
`git`	Required by `squash-tf-services` which may clone SCM repositories

No local browser inside the container — Selenium runs in perfshop-selenium, the test runner is a WebDriver client only.

Python dependencies — two pedagogical profiles¶

RUN pip install --no-cache-dir \
    robotframework==7.0 \
    robotframework-seleniumlibrary==6.3.0 \
    robotframework-requests==0.9.6 \
    robotframework-jsonlibrary==0.5 \
    pytest==8.0 \
    selenium==4.18.1 \
    requests==2.31.0 \
    allure-robotframework \
    squash-tf-services

The Dockerfile comment is explicit: "QA dependencies — two student profiles. Beginner profile: robotframework + SeleniumLibrary + RequestsLibrary + JsonLibrary. Advanced profile: pytest + selenium".

Profile	Libraries	Typical usage
Beginner	Robot Framework 7.0 + SeleniumLibrary 6.3.0 + RequestsLibrary 0.9.6 + JsonLibrary 0.5	Tests written in keyword-driven (table-driven) syntax, easy to read without Python experience
Advanced	pytest 8.0 + selenium 4.18.1 + requests 2.31.0	Tests written in Python, fixtures, parameterization, fine-grained control

Both profiles coexist in the same container — nothing prevents a student from starting with Robot Framework and then gradually migrating to pytest as their skill grows.

Two complementary dependencies:

allure-robotframework: generates Allure reports from Robot Framework results — can be used by students who want to experiment with an alternative visual reporting.
squash-tf-services: client-side services needed for the test runner to be compatible with Squash TF orchestration (automatic SCM clone, structured result return).

Pinned versions¶

All versions are pinned in the Dockerfile. This is intentional — reproducibility is essential for pedagogical use: a student who learns with RF 7.0 must find exactly the same behavior the next day, and an instructor must be able to rely on documented behaviors.

OpenSSH configuration¶

RUN mkdir -p /var/run/sshd && \
    echo 'root:perfshop' | chpasswd && \
    sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config && \
    sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config

Three modifications:

Action	Effect
`echo 'root:perfshop' \\| chpasswd`	Sets the root password to `perfshop` (for the orchestrator)
`PermitRootLogin yes`	Allows direct SSH login as root
`PasswordAuthentication yes`	Allows password authentication (disabled by default on recent Debian images)

Root access by password — strictly pedagogical context

The Dockerfile comment is explicit: "Root access with password 'perfshop' — pedagogical use only. The container is not exposed externally (internal Docker network)". In a real production deployment (even internal), it would be necessary to switch to SSH key authentication and create a dedicated non-root user. For PerfShop, this configuration is acceptable because the container exposes no port to the host (port 22 is internal), and the goal is pedagogical.

Entrypoint and variable propagation¶

COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

The Dockerfile comment explains: "sshd does not inherit Docker variables. docker-entrypoint.sh writes /etc/environment at startup so that robot and pytest see them in SSH sessions launched by the orchestrator".

This is a classic pitfall of SSH integration in a Docker container: when sshd accepts an incoming connection and forks a shell, that shell does not inherit the environment variables set by Docker Compose (they are only in the PID 1 process). The entrypoint works around this by writing all useful variables into /etc/environment, which is read by pam_env on every login.

Concretely, the variables SELENIUM_REMOTE_URL, PERFSHOP_FRONTEND_INTERNAL, PUBLIC_API_URL, and PERFSHOP_LANG are thus available both in a docker exec session and in an SSH session — guaranteeing consistent behavior between the two execution channels.

The service in compose¶

perfshop-test-runner:
  image: perfshop-test-runner:latest
  build:
    context: ./test-runner
    dockerfile: Dockerfile
  container_name: perfshop-test-runner
  environment:
    - SELENIUM_REMOTE_URL=http://perfshop-selenium:4444/wd/hub
    - PERFSHOP_FRONTEND_INTERNAL=${PERFSHOP_FRONTEND_INTERNAL:-http://perfshop-frontend:80}
    - PUBLIC_API_URL=${PUBLIC_API_URL:-http://localhost:8080}
    - PERFSHOP_LANG=${PERFSHOP_LANG:-fr}
  volumes:
    - ./test-runner/scripts:/scripts
    - ./test-runner/logs:/rf-logs
  entrypoint: ["/usr/local/bin/docker-entrypoint.sh"]
  command: ["/usr/sbin/sshd", "-D", "-e"]

Command and entrypoint¶

The entrypoint writes /etc/environment then launches /usr/sbin/sshd -D -e:

Flag	Effect
`-D`	Daemon mode — stays in foreground (required so the container does not exit)
`-e`	Logs to stderr (instead of syslog) — captured by Docker and therefore by Promtail / Vector

The container does nothing else but run sshd and wait for connections (SSH or docker exec). This is the same principle as perfshop-jmeter which runs tail -f /dev/null while waiting for shots — the container has no "business service" of its own, it only serves as an execution substrate.

Propagated environment variables¶

Variable	Role	Visibility
`SELENIUM_REMOTE_URL`	WebDriver hub URL	Used by Robot Framework and pytest to open a remote browser
`PERFSHOP_FRONTEND_INTERNAL`	Internal frontend URL	Tests navigate to `http://perfshop-frontend:80` (Docker DNS)
`PUBLIC_API_URL`	Public backend URL	Useful if a test asserts against a canonical URL
`PERFSHOP_LANG`	Current language	Lets a test pick the right i18n selectors (FR or EN)

Volumes¶

Volume	Mount point	Role
`./test-runner/scripts` (bind mount)	`/scripts`	Shared test scripts — volume read by the test runner (execution) and written by forgejo-seed (initial push of scripts to the Git repository)
`./test-runner/logs` (bind mount)	`/rf-logs`	Logs and reports — written by Robot Framework and pytest on every run, read by Promtail (Loki) and Vector (OpenSearch)

The WORKDIR in the Dockerfile is /scripts, so any docker exec or SSH execution starts directly in the scripts directory.

The two execution modes¶

Mode 1 — `docker exec` from Scripts UI¶

sequenceDiagram
  autonumber
  participant SUI as perfshop-scripts-ui<br/>(Node.js)
  participant SOCK as /var/run/docker.sock
  participant TR as perfshop-test-runner
  participant SEL as perfshop-selenium

  SUI->>SOCK: POST /containers/perfshop-test-runner/exec<br/>{cmd: ["robot", "--outputdir", "/rf-logs/...", "login.robot"]}
  SOCK->>TR: exec in /scripts
  TR->>TR: Reads variables from /etc/environment
  TR->>SEL: POST /wd/hub/session
  SEL-->>TR: Chrome session
  TR->>SEL: WebDriver commands
  TR->>TR: Writes log.html, report.html, output.xml in /rf-logs/
  TR-->>SUI: Exit code + stdout

Advantages: - No credentials to manage on the Scripts UI side (uses the Docker socket) - Fast and lightweight

Mode 2 — SSH from Squash Orchestrator¶

sequenceDiagram
  autonumber
  participant ORCH as perfshop-orchestrator
  participant TR as perfshop-test-runner (sshd)
  participant SEL as perfshop-selenium

  ORCH->>TR: SSH root@perfshop-test-runner<br/>(password perfshop)
  TR->>TR: pam_env reads /etc/environment
  ORCH->>TR: SCP script.robot → /scripts/tmp/
  ORCH->>TR: exec "robot --outputdir /rf-logs/... script.robot"
  TR->>SEL: POST /wd/hub/session
  SEL-->>TR: Chrome session
  TR->>TR: Runs the test
  TR-->>ORCH: Exit code + output.xml
  ORCH-->>ORCH: Parses output.xml and returns to Squash TM

Advantages: - Compatible with the distributed Squash TF architecture (the runner could theoretically sit on another machine) - Structured result format (output.xml) directly parsable by Squash TM

No port exposure¶

# No "ports:" in the perfshop-test-runner block

The container exposes no port to the host. Port 22 is only reachable from the internal Docker network — the orchestrator connects to it through perfshop-test-runner:22. No external user can reach sshd.

Logs collected by two pipelines¶

Files written to /rf-logs/ are collected simultaneously by Promtail and Vector:

Promtail → Loki: the rf-runner-log Promtail job tails /rf-logs/*.log files (see ../observability/loki.md)
Vector → OpenSearch: Vector collects container logs from perfshop-test-runner through its Docker source (sshd stdout + stderr)

These two streams are complementary: - Loki is optimal for "find all executions of the login.robot test in the last 10 minutes" - OpenSearch enables structured aggregations such as "how many PASS vs FAIL tests per day over the last week"

Image not pullable from a public registry¶

The perfshop-test-runner:latest tag is local to the build host. No push to Docker Hub or any other registry is performed. build.sh runs docker build on the test-runner/ folder at every deployment. This aligns with the PerfShop philosophy: everything is reproducible from source, with no dependency on a third-party registry.

Going further¶

Selenium — the WebDriver engine driven by the test runner
Squash Orchestrator — one of the two triggers
Scripts UI — the other trigger (via docker exec)
Loki — collection pipeline for /rf-logs/*.log logs
OpenSearch — alternative pipeline through Vector