Overcast cluster architecture

Distributed systems can become very complicated very quickly forcing developers to learn heaps of new concepts, rewrite lots of code and eventually becoming very hard to manage.

The Overcast Cluster's architecture was intentionally kept as simple as possible to avoid overwhelming developers with excessive complexity, while introducing many essential features to make it flexible and versatile.

As briefly mentioned in the overview, at its core, an Overcast Cluster consists of two types of nodes:

  • Lobby Node:
    • Acts as the main entry point for all clients.
    • Manages the Game Nodes by monitoring the load and resizing their number as needed.
    • Handles the Load Balancing logic.
    • Manages players match-making, finding available games or starting new ones on the available nodes.
    • Provides cluster management via an extended SFS2X AdminTool.
  • Game Node:
    • Game Nodes are the main engine of the SFS2X Cluster.
    • Handle all of Game Rooms and relative custom Extension code.
    • Can dynamically or manually join and leave the cluster.

These nodes are all SmartFoxServer 2X instances with a few added components that determine their role in the cluster.

In addition to these features you also get: customizable Load Balancing logic, advanced cluster-wide match-making, new monitoring and management tools via an extended AdminTool, updated SFS2X client side API with new events and requests and tons more, which will be discussed in these tutorials.

Multi-game cluster example

To give you a clearer idea of how the Overcast Cluster works we discuss a multi-game use case from a high level and some of its implementations details. You'll find more of these use case examples throughout this section of the documentation.

In this example we are building a multiplayer card game service where users can play a number of card games, such as Texas Hold'em, Gin Rummy and Pinochle. The system is designed as shown in the diagram below with a central Lobby Node for all players, multiple Game Nodes that will scale up/down depending on the load and a Database Server for persistence.

The whole cluster is deployed as a single Overcast App, which we can setup from the Overcast HQ interface in a few clicks (you will learn more about this in the next article from this section).

In order to have a better high-level understanding of this system we're going to discuss the two sides (server and client) separately, starting from the server side.

Server side perspective

Since all our players initially login on the Lobby Node, we need an SFS2X Extension that validates their access against the database or allows to create a new account and then login. It may not come as a surprise that this whole set of operations don't require any new knowledge and can be implemented in the same way it would be done with a non-cluster version of SFS2X.

The signup and login processes can be setup with a custom Extension or using the more convenient Login Assistant and SignUp Assistant provided by the SFS2X API.

Let's now take a quick look at what other activities will be taken care of by the Lobby:

  • Buddy Management: players joining the Lobby can chat and manage their buddy lists, either by searching for new friends or by adding buddies they've been playing with.
  • Profile Management: users can check and modify their profile status, avatar, gameplay statistics etc; all these features can be easily implemented via Zone Extension and the cluster's database.
  • Matchmaking: players are matched via MatchExpressions which allow great flexibility, just like in standard SFS2X. The Overcast Cluster provides new client and server API to make it work smoothly in a multi-server system.
NOTE: besides the Buddy Management, we don't recommend running other user interaction on the Lobby Node. For example Chat Rooms are not a good fit for the Lobby, as they can quickly become resource intensive and we want to keep any Room-based interaction on the Game Nodes.

Now we can take a look at the the Game Nodes.

These servers are dynamic, meaning that they are actively monitored by the Lobby and added or removed based on the current cluster load. We discuss these scalability mechanisms in depth in the Advanced Clustering section of this documentation.

Every node uses three separate Room Groups to organize games by type and each Game Room runs its own Extension that manages each game instance.

Client side perspective

The client application is built very similarly to non-cluster apps but with a twist: it can use multiple connections towards different cluster nodes, depending on the design of the game application itself.

Regardless of the specifics of each use case, a common trait to all apps is the starting connection to the Lobby.

As we mentioned earlier, once inside the Lobby we can load our buddy list, interact with our friends and communicate with the Lobby Extension to manage profiles, search for other players and most importantly jump into a game via match-making.

Being a high-level overview we're not going dive deep in the implementation details, as there will be plenty of specific examples in the next articles.

However we can take a sneak peek at how client side match-making is done for our cluster multi-game app:

         MatchExpression matchExp = new MatchExpression("gameStarted", BoolMatch.EQUALS, false);
         ClusterRoomSettings newRoom = new ClusterRoomSettings("NewGame");
         newRoom.IsGame = true;
         newRoom.MaxUsers = 4;
         ...
         
         lobby.Send(new ClusterJoinOrCreateRequest(matchExp, new List<string>(){"Gin Rummy"}, newRoom));
         var matchExp = new MatchExpression("gameStarted", BoolMatch.EQUALS, false);
         var newRoom = new ClusterRoomSettings("NewGame");
         newRoom.setGame(true);
         newRoom.setMaxUsers(4);
         ...
         
         lobby.send(new ClusterJoinOrCreateRequest(matchExp, Arrays.asList("Gin Rummy"), newRoom));

Here we want to match any Game Room that has the RoomVariable gameStarted set to false and search in the "Gin Rummy" Group only. Remember? We organized the Zone in multiple Room Groups for each card game available.

We also pass the settings for a new Game Room in case nothing is found so that a new game can be started and joined immediately.

While this looks like a standard, non-cluster SFS2X match-making request there are a number of other operations going on in the Lobby when this is executed:

  • The Lobby invokes the internal Load Balancer and selects a Game Node to run the match-making.
  • The Game Node runs the match-making request and tells the Lobby which Room should be joined.
  • The client receives an event called CONNECTION_REQUIRED passing the client the info to start a new connection go the Game Node.
  • Finally the client connects to the Game Node and enters the game.

This is how the CONNECTION_REQUIRED event looks like:

         lobby.AddEventListener
         (
            SFSClusterEvent.CONNECTION_REQUIRED, 
            (BaseEvent evt) => {
               ConfigData cfg = (ConfigData)evt.Params["configData"];
               String username = (string)evt.Params["userName"];
               String password = (string)evt.Params["password"];
               
               SmartFox game = new SmartFox();
               
               // Other event listeners
               ...
               
               game.Connect(cfg);
            }
         );
         lobby.addEventListener
         (
            SFSClusterEvent.CONNECTION_REQUIRED, 
            (BaseEvent evt) -> {
               var cfg = evt.getParams().get("configData");
               var username = evt.getParams().get("userName");
               var password = evt.getParams().getParams("password");
               
               var game = new SmartFox();
               
               // Other event listeners
               ...
               
               game.connect(cfg);
            }
         );

We obtain the parameters for the connection to the Game Node directly from the event parameters. All we need to do is create a new instance of the SmartFox class (above called game), pass the ConfigData object to its Connect()connect() method and we're done.

Also we need to store the username and password provided by the event and we'll use in the LoginRequest once the connection is established (as usual with any SFS2X connection).

And this is the state of the client connected to both the Lobby and Game Node.

Multi-connection management

With two (or more) active client connections we can simultaneously listen to multiple server events and build even more complex interactions. For instance we could implement a buddy list panel with chat and contacts that is always present, both inside and outside of games allowing player to interact at all times.

Other games could allow players to participate in multiple matches at the same time. For example: a chess game where users can challenge several opponents at once.

However not all use cases are suitable for this approach (e.g. real-time games) and ultimately the approach to connection management is left to the developer.

Locality of data

One of the strengths of Overcast Clustering is that it maintains its data local to each server. This means that the state of one server is not shared with other nodes and this is important to keep things simple enough. When state gets shared across multiple nodes you're instantly teleported in a parallel universe: that of distributed computing, which is a very complex subject requiring lots of specific expertise.

In other words the Overcast Cluster allows you to keep your Extension development as simple as it is with a standalone SmartFoxServer while offering great horizontal scalability and clustering features.

Additionally you don't have to give up shared state. It can be easily implemented via a central database server, which has a predictable performance and it is much easier to integrate.

In any case, for now, don't worry about these concepts we'll return to this topic in the sections dedicated to cluster server-side development.

Take-away concepts

Before wrapping up, let's recap the principal aspects of the Overcast Cluster architecture:

  • The Lobby is the central access to the cluster for all clients
    • Both login and sign-up operations are handled here, maybe with the help of SignUp and Login Assistant provided by the SFS2X API or with a custom implementation.
    • The Lobby should deal mostly with user-to-backend interaction (e.g. profile/inventory management), buddy lists and match-making, while leaving the user-to-user interaction to Game Nodes.
    • In the background the Lobby provides Load Balancing and Node Orchestration for the whole cluster.

  • Game Nodes split the load of all active games, handling any number of Game Rooms
    • Rooms can be organized in Room Groups or 'tagged' via Room Variables and later filtered via regular Match Expressions.
    • Game Rooms run their own Extensions to manage the game logic.
    • Chat Rooms (public and private) can also be hosted on Game Nodes.