• MZBench is a generic load testing tool which uses bdl files to describe a load.
• VMQ_MZBench is an extension (aka worker) to MZBench allowing you to load test an MQTT broker.
• MZBench source code is too old to build on modern OS (Python 2 and other issues) and the official Docker image is too old to support VMQ_MZBench (don't ask me how I know)
• An alternative Docker file works great. The incantation to run MZBench after the Docker container is built is docker run --network="host" -d -p 4800:4800 mzbench
• The alternative docker file runs a listener on 0.0.0.0:4800 so point your brower to the load tester URL on port 4800 and you are good to go
• You do not need to install VMQ_MZBench as it will be installed automatically by the bdl code.
• Example bdl code for VMQ_MZBench

The easiest way to use it is to log into the Docker container and create and run the bdl files from the command line. Then read the results in the web interface

• Log into container:
  • find out container id: docker ps
  • log into shell: docker exec -it <CONTAINER_ID> /bin/bash
• run test. Example:

cd /app/mzbench
./bin/mzbench run examples.bdl/ramp.bdl

I wrote filiminSwarm2.py which emulates groups of lamps. Because of a limitation in python and/or the mqtt implementation of python (183, 238, 499) the script can only connect 340 lamps in a single Python instance. In theory a recompile of Python2 should fix this but I tried and with the FD_SETSIZE changed from 1024 to 65536 I got buffer overflow errors in Python after 340 connections. So to work around this I also wrote a Bash script that spawns multiple instances of the Python script (filiminSwarms2.sh).

• filiminSwarms2.sh was able to create 58 swarms of 340 lamps each, grouped in 3s with each lamp in group touched every 120 to 150 seconds (filiminSwarms2.sh 58 3 120 150 0) on an 8GB 4CPU Linode VM. That's a total of 19,720 lamps. At 67 swarms the ephemeral port range appears to become an issue
• The current Linode haproxy load balancer (haproxyMaster) is 8GB/4 dedicated CPUs. It successfully handled the load of 11 load testers (11 * 19720 = 216920 lamps). There were 12 mosquitto nodes connected to it at less than 10% CPU. Sync service was around 50-60% CPU.

The default ip_local_port_range severely limits the number of connections between 2 machines.

• current range: sysctl -A | grep ip_local_port_range
• to extend without rebooting edit /etc/sysctl.conf (good values are 15000 64000) then sysctl -p /etc/sysctl.conf .
• effective range: sysctl net.ipv4.ip_local_port_range