» Performance Testing, Part 3

In part one of this series we've discussed how to implement a simple stress test application in Java, targeting the native SFS2X socket protocol.

In part two we have ported the same application to Javascript using WebSocket and targeting the Firefox browser, for performance reasons.

Finally, in this last chapter, we are going to discuss testing tips, best practices and how to avoid potential pitfalls.

» Performance Notes (Java)

When replicating many hundreds / thousands of clients we should keep in mind that every new instance of the SmartFox class (the main API class) will use a certain amount of resources, namely RAM and threads.

For simple test clients each instance should take ~1MB of heap memory which means we can expect 1000 clients to take approximately 1GB of RAM. In this case you will probably need to adjust the heap settings of the JVM by adding the -Xmx switch to the startup script.

Similarly the number of threads in the JVM will increase by 2 for each new client generated. So, for example, a test with 1000 clients will end up using 2000 threads, which is already a pretty high value.

Any relatively modern machine (e.g 4-6 cores, 8GB RAM) should be able to run at least 1-2000 clients, although the complexity of the client logic and the rate of network messages might reduce this value.

» Performance Notes (Javascript)

In part two of this series we have already discussed some of the performance aspects of testing in the browser. In particular, we found significant limitations on the number of concurrent connections generated in a single web application and we've found Mozilla Firefox to be the most flexible for this sort of tests.

» Essential tips for testing

» Advanced testing

1) Login: in our simple example we have used an anonymous login request and we don’t employ a server side Extension to check the user credentials. Chances are that your system will probably use a database for authentication and you might want to test how the DB performs with a high traffic.

A simple solution is to pre-populate the user’s database with index-based names such as User-1, User-2 … User-N. This way you can build a simple client side logic that will generate these names with an auto-increment counter and perform the login. Passwords can be handled similarly using the same formula, e.g. Password-1, Password-2… Password-N

TIP: When testing a system with an integrated database always monitor the Queue status under the AdminTool > Dashboard. Slowness with DB operations will likely show up in those queues, as threads become less efficient in dealing with the requests.

2) Joining Rooms: another problem for automated tests is how to distribute clients in multiple Rooms. Suppose we have a game for 4 players and we want to distribute 1000 clients into game Rooms for 4 players. A simple solution is to create this logic on the server side.

The Extension will take a generic “join” request (via ExtensionRequest) and perform a bit of custom logic:

A similar example has been discussed in details in this post in our support forum.

» Further resources

There are two more parts to this article series: