This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | |
| lwc:mqtt:loadtesting [2021/10/05 09:13] – [Filimin Swarms] John Harrison | lwc:mqtt:loadtesting [2021/10/06 13:18] (current) – [Filimin Swarms] John Harrison |
|---|
| I wrote ''filiminSwarm2.py'' which emulates groups of lamps. Because of a limitation in python and/or the mqtt implementation of python ([[https://github.com/eclipse/paho.mqtt.python/issues/183|183]], [[https://github.com/eclipse/paho.mqtt.python/issues/238| | I wrote ''filiminSwarm2.py'' which emulates groups of lamps. Because of a limitation in python and/or the mqtt implementation of python ([[https://github.com/eclipse/paho.mqtt.python/issues/183|183]], [[https://github.com/eclipse/paho.mqtt.python/issues/238| |
| 238]], [[https://github.com/eclipse/paho.mqtt.python/issues/499|499]]) the script can only connect 340 lamps in a single Python instance. In theory a recompile of Python2 should fix this but I tried and with the ''FD_SETSIZE'' changed from 1024 to 65536 I got buffer overflow errors in Python after 340 connections. So to work around this I also wrote a Bash script that spawns multiple instances of the Python script (''filiminSwarms2.sh''). | 238]], [[https://github.com/eclipse/paho.mqtt.python/issues/499|499]]) the script can only connect 340 lamps in a single Python instance. In theory a recompile of Python2 should fix this but I tried and with the ''FD_SETSIZE'' changed from 1024 to 65536 I got buffer overflow errors in Python after 340 connections. So to work around this I also wrote a Bash script that spawns multiple instances of the Python script (''filiminSwarms2.sh''). |
| * ''filiminSwarms2.sh'' was able to create 66 swarms of 340 lamps each, grouped in 3s with each lamp in group touched every 5 to 7 seconds (''filiminSwarms2.sh 66 3 5 7 0'') on an 8GB 4CPU Linode VM. That's a total of 22,440 lamps. At 67 swarms the ephemeral port range appears to become an issue | * ''filiminSwarms2.sh'' was able to create 58 swarms of 340 lamps each, grouped in 3s with each lamp in group touched every 120 to 150 seconds (''filiminSwarms2.sh 58 3 120 150 0'') on an 8GB 4CPU Linode VM. That's a total of 19,720 lamps. At 67 swarms the ephemeral port range appears to become an issue |
| * 45 swarms (15,300 lamps) ran successfully on 4GB 2CPU VM | * The current Linode haproxy load balancer (haproxyMaster) is 8GB/4 dedicated CPUs. It successfully handled the load of 11 load testers (11 * 19720 = 216920 lamps). There were 12 mosquitto nodes connected to it at less than 10% CPU. Sync service was around 50-60% CPU. |
| |
| ===== Ephemeral port range ===== | ===== Ephemeral port range ===== |