ServerConcepts

Nexus Server Concepts

You should be familiar with the concepts in ClientServerConcepts as this documentation will refer freely to ideas and terms defined there.

Entities and Execution Contexts

A Nexus server is most fundamentally a collection of entities which are associated with execution contexts. Entities dictate the threading model of the server, and the form the basis on which NexusObjects are hosted and NexusServices are implemented. They also form the foundation on which a Nexus system can be spread across multiple servers (details in ServerServerConcepts).

An entity is a plain-old Java object (POJO) which is bound to an execution context. An execution context is essentially a queue of actions that will be processed serially. Two threads will never execute actions in a particular execution context at the same time. This allows the code for an entity to have a "locally single threaded" view of the world, and to avoid worrying about multithreading issues except when communicating between entities/execution contexts.

The combination of entities and execution contexts is very similar to the actor model for concurrent programming, but there are a couple of important differences. First, multiple entities can be bound to the same execution context, allowing these entities to interact with one another directly, without concern for threading issues. Second, instead of defining messages that are exchanged between actors, Nexus Entities encapsulate actions to be taken on entities into closures that are executed in the target entity's execution context.

This is most easily explained with some code (note that Singleton is a marker interface for singleton entities):

class MailManager implements Singleton {
  void sendMail (String address, String subject, String body) { ... }
}

Nexus nexus = ...;
final String addr = ..., subj = ..., body = ...;
nexus.invoke(MailManager.class, new Action<MailManager>() {
  public void invoke (MailManager mailMgr) {
    mailMgr.sendMail(addr, subj, body);
  }
});

The body of the Action will be run in the execution context to which the (singleton) EmailManager instance is bound.

Nexus Action and Request invocations are currently a bit on the verbose side, due to the syntactic ceremony required to create an anonymous inner class. However, when Java 8 is widely available, the new Java 8 closure syntax will substantially simplify things. The above example will be reduced to:

String addr = ..., subj = ..., body = ...;
nexus.invoke(MailManager.class, mm => mm.sendMail(addr, subj, body));

which is far more compact and readable. For the remainder of this documentation, we will use Java 8 syntax as it allows us to avoid bloating our example code with inner class boilerplate and instead to focus on the semantics of the system.

Actions and Requests

There are two ways to interact with an entity: actions and requests.

• Action, which we saw above, simply packages up a closure and executes it in the execution context of the target entity.
• Request packages up a closure, executes it in the execution context of the target entity, captures the return value and delivers it back to the calling execution context. The calling execution context will block while the Request is processed.

Here is an example of code using Request:

public class UserManager implements Singleton {
  public String getEmail (int userId) { ... }
}

public class MailManager implements Singleton {
  private Nexus nexus = ...;

  public void sendMail (int userId, String subj, String body) {
    String addr = nexus.request(UserManager.class, um => um.getEmail(userId));
    sendMail(addr, subj, body);
  }

  protected void sendMail (String addr, String subj, String body) { ... }
}

In the above example, the execution context of the MailManager will block while the call to UserManager.getEmail is being processed. We discuss in more detail the benefits and tradeoffs of this blocking approach in the Entity Design and Blocking section below. In situations where one wishes to issue numerous requests in parallel and then aggregate their results, as well as in cases where one requires control over request timeouts, requestF can be used to obtain a Future:

public class UserManager implements Keyed {
  private final Integer userId = ...;
  public Comparable<?> getKey () {
    return userId;
  }
  public String getEmail () { ... }
}

public class MailManager implements Singleton {
  private Nexus nexus = ...;

  public void sendMail (List<Integer> userIds, String subj, String body) {
    // issue all of our email requests in parallel
    List<Future<String>> addrs = new ArrayList<>();
    for (Integer userId : userIds) {
      addrs.add(nexus.requestF(UserManager.class, userId, um => um.getEmail()));
    }
    // now block waiting for them to complete, and make use of the results
    for (Future<String> addr : addrs) {
      sendMail(addr.get(), subj, body);
    }
  }

  protected void sendMail (String addr, String subj, String body) { ... }
}

Note that in this second scenario, the UserManager is structured such that each user has their own execution context (it is a Keyed entity, which is described in the next section). We dispatch a request to each execution context to obtain the email. If the UserManager were structured as it was in the first example, then we would simply be issuing multiple requests to the same execution context, and those requests would be processed serially by that context. No parallelism would be achieved. It only makes sense to parallelize requests if they are being dispatched to different execution contexts.

Singleton, Keyed and NexusObject entities

Entities come in three main forms:

• Singleton entities, for which only a single instance exists per server.
• Keyed entities, for which many instances exist, each with a unique identifying key. Only a single keyed entity for a given key will exist in the entire network and it will reside on one server in the network.
• NexusObject entities, which can be referenced by Address and which may also be a singleton or keyed entity (though a keyed NexusObject entity would be uncommon).

We'll talk about entities in the context of a single server system, which is sufficient to understand their main properties. See ServerServerConcepts for additional details on how things differ in a multi-server system.

Keyed entities

Keyed entities are the main workhorse of a Nexus system. The ability to scale your system (both across many cores of a single server and across many servers in a multi-server configuration) will rely on a thoughtful decomposition of your system into keyed entities.

One natural design pattern is to create a keyed entity to manage services for a particular user. This allows data private to a given user to be manipulated on its own execution context, without blocking any other users. When the user reads or modifies their private state, their execution context can do things like talk to a database to make those changes without blocking other users' execution contexts and while still providing good response times to the user in question.

Multi-user distributed systems will almost always involve situations where users interact (multiplayer game systems certainly will). This is another sensible place to introduce a keyed entity. Take for example a server that allows users to play poker. When a group of players starts a game, a keyed entity can be created to manage the game's state and behavior. In a case like this, one will usually assign an arbitrary, but unique key to the entity to differntiate it from other games. A monotonically increasing int suits this purpose.

The game will naturally have internal state, which can be managed by the game entity without concern for threading issues, because the entity is effectively single threaded with regard to its own internal state. The game can make inline blocking calls to read and write to a database, or to request information from other entities, without concern that it is holding up unrelated services or that it will be held up by unrelated services.

With this structure, a server can easily scale to manage hundreds of games, the processing for which will be multiplexed across the CPU cores provided by that server. If the system becomes extremely popular, it can then be scaled across multiple servers, with each server handling a subset of the games. Because of the "server agnosticism" of the client API, clients need not be specially structured to anticipate this transition from single to multiple servers.

Registering Keyed entities with the Nexus

A keyed entity can be registered with the Nexus explicitly, or it can be created on demand. A scenario where explicit registration is used might look like the following:

public class GameManager implements Keyed {
  private final Integer gameId;

  public GameManager (int gameId) {
    this.gameId = gameId;
  }

  public Comparable<?> getKey () { return gameId; }

  public void start (Player[] players) { ... }
}

public class LobbyManager implements Singleton {
  private Nexus nexus = ...;

  public void startGame (Player[] players) {
    int gameId = nextGameId();
    nexus.registerKeyed(new GameManager(nexus, gameId));
    // tell the game manager to start its new game, but have it do the "start game" processing on
    // its own execution context
    nexus.invoke(GameManager.class, gameId, gm => gm.start(players);
  }

  protected int nextGameId () { ... }
}

Here, the LobbyManager assigns a key to a new game and creates a GameManager (a Keyed entity) to handle the game state and behavior. The lobby manager hands off the players in question to the game manager which then handles the operation of the game on its own execution context. See this example code for an actual working example of a lobby and game manager.

Keyed entities can also be registered on demand. This can be useful for situations like creating an entity to manage a user's state, where the key for the entity is known in advance, rather than created by some managing entity, and where you may prefer not to have a central place where these entities are created. An example:

public class UserManager implements Keyed {
  private final Integer userId;

  public UserManager (int userId) {
    this.userId = userId;
  }

  @Override public Comparable<?> getKey () {
    return userId;
  }

  // this will do the appropriate thing whether the user is logged on or not
  public void sendMessage (int fromUserId, String text) { ... }
}

// during server initialization, register a factory for our UserManager
nexus.registerKeyedFactory(UserManager.class, new KeyedFactory<UserManager>() {
  public UserManager create (Nexus nexus, Comparable<?> key) {
    return new UserManager((Integer)key);
  }
});

// where we wish to deliver a message, simply invoke an action and the appropriate UserManager will
// be auto-created if it does not already exist
int toUserId = ..., fromUserId = ...;
String text = ...;
nexus.invoke(UserManager.class, toUserId, um -> um.sendMessage(fromUserId, text));

Singleton entities

Singleton entities, as their name suggests, exist only as a single instance. As such, they often do not manage any state, but rather exist to provide access to global services. Or in the case of systems that are not expected to scale arbitrarily, they may manage some global state which is known to fit comfortably into a single server with a modest number of users.

One example of a singleton entity is an authorization service, which ensures that a user has appropriate credentials before giving them access to other parts of a system. A LoginManager might provide a LoginService which accepts a user's credentials and either handles login directly (in the case where a system is known to be small enough that a single thread can handle the login traffic) or it can then create a UserManager with the candidate credentials and allow the UserManager to process the login on a separate execution context for that user. The second example in the Entity Design and Blocking section shows code for such an example.

Registering Singleton entities with the Nexus

Singleton entities are registered using only their class token as an identifier. This registration would generally happen during the server initialization process so that the singleton entity is available immediately. For example:

public class LoginManager extends Singleton {
  public LoginManager (Nexus nexus) { ... }
}

public class MyServer {
  public static void main (String[] args) {
    ExecutorService exec = Executors.newCachedThreadPool();
    NexusConfig config = ...;
    NexusServer server = new NexusServer(config, exec);
    server.registerSingleton(new LoginManager(server));
    // ...
  }
}

Another common pattern is for a Singleton entity to have an associated singleton NexusObject which makes bootstrap distributed services available to clients. This example is shown below in the section on NexusObject entities.

NexusObject entities

Entities are a server-side concept and have no correlate in client code (a Nexus client is expected to be effectively single threaded, as explained in the Event Dispatch Thread section of ClientServerConcepts). However, the client world and the server world meet in two places: NexusObject and distributed service implementations.

A NexusObject can have is own execution context and be registered as a Keyed or Singleton entity, or it can be piggybacked on the execution context of an existing entity. The latter approach is most common, because one almost always has distributed services associated with a distributed object, and those services are most sensibly implemented by a server-only entity.

Here's an example of how distributed objects are often used in conjunction with server entities:

public class Coord implements Streamable {
  public final int x, y;
  public Coord (int x, int y) {
    this.x = x;
    this.y = y;
  }
  // equals and hashCode implementation required!
}

public interface TicTacToeService extends NexusService {
  void play (Coord coord);
}

public class TicTacToeObject extends NexusObject {
  public final DService<TicTacToeService> svc;
  public final DMap<Coord,Integer> plays = DMap.create(this);
  public TicTacToeObject (DService.Factory<TicTacToeService> svc) {
    this.svc = svc.createService(this);
  }
}

public class TicTacToeManager implements Keyed, TicTacToeService {
  private final Integer gameId;
  private final TicTacToeObject obj;

  public TicTacToeManager (Nexus nexus, int gameId) {
    this.gameId = gameId;
    // register ourselves as a keyed entity
    nexus.registerKeyed(this);
    // create and register our distributed object as a child entity, which means
    // that it shares our execution context
    obj = new TicTacToeObject(Factory_TicTacToeService.createDispatcher(this));
    nexus.register(this, obj);
  }

  @Override public Comparable<?> getKey () { return gameId; }
  @Override public void plain (Coord coord) { ... }
}

The TicTacToeManager is a keyed entity, which establishes an execution context for the game. It then creates and registers a distributed object (TicTacToeObject) as a child entity, which causes the distributed object to share the same execution context as the manager. This means that events that arrive on the distributed object will be dispatched in the manager's execution context. So the manager can safely add listeners to the distributed object without concern for threading issues as the distributed object is part of its little single threaded world.

Because the TicTacToeService is published via the TicTacToeObject, calls to TicTacToeService methods will be dispatched in the execution context of the TicTacToeObject, which is the same as the context of TicTacToeManager. It's critical to wire this up properly, otherwise you'll have methods being called on an entity from the wrong context. Fortunately this natural pattern of sharing an execution context between a manager entity and a distributed object results in everything being dispatched correctly.

Entity Design and Blocking

Being able to block awaiting the response of another entity is very useful in preserving linear, comprehensible code that is not scattered across myriad callbacks. It is also helpful in eliminating a whole class of "async" bugs. Even in a single threaded application, if you suspend a computation in an Entity while waiting for some third party to get back to you with a result, but then you allow new computations to start executing in that entity while the other computation is waiting, you can find yourself in situations where those computations conflict, or where the suspended computation resumes and things that were obviously true when the computation was suspended are no longer true. By blocking the entire entity while it waits for an asynchronous result, you are ensured that no one is going to come in and change things out from under you.

However, this means that one must take care not to create bottleneck entities, through which many computations flow. For example, you may think to create a singleton entity UserManager which loads user information from a database when they log into the sytem and tracks which users are logged in. A brief outline of such code might look like the following:

public class UserManager implements Singleton {
  private UserDatabase db = ...;
  private Map<String,User> users = new HashMap<String,User>();

  public User resolveUser (String username) {
    User user = users.get(username);
    if (user == null) {
      user = db.loadUser(username); // slow blocking call
      users.put(username, user);
    }
    return user;
  }
}

public class LoginManager implements Singleton, LoginService {
  private Nexus nexus = ...;

  // from LoginService
  public void authenticate (final String username, String passHash,
                            Callback<Address<UserObject>> callback) {
    User user = nexus.request(UserManager.class, new Request<UserManager,User>() {
      public User invoke (UserManager userMgr) {
        return userMgr.resolveUser(username);
      }
    });
    if (!user.checkPassword(passHash)) {
      // report failure to callback; tell user manager to unload user record now or after a short
      // timeout so that we can reuse the loaded user record for successive auth attempts
    } else {
      // create UserObject entity, register with Nexus, pass address to callback
    }
  }
}

This results in all authentication requests queueing up on the UserManager execution context, waiting for it to load the user information from the database. To avoid this sort of bottleneck, we want to establish an execution context for an authenticating user as early as possible. Instead of having a UserManager singleton, we can instead use a Keyed entity and have a manager per user, each with their own exection context.

public class UserManager implements Keyed {
  private final Nexus nexus = ...;
  private final String username;
  private UserDatabase db = ...; // could get via dependency injection, or Nexus
  private User user;

  public UserManager (Nexus nexus, String username) {
    this.nexus = nexus;
    this.username = username;
  }

  @Override public Comparable<?> getKey () {
    return username;
  }

  public void authenticate (String passHash, Callback<Address<UserObject>> callback) {
    user = db.loadUser(username); // slow blocking call
    if (!user.checkPassword(passHash)) {
      // report failure to callback; either unregister ourselves immediately, or set a timer
      // to unregister ourselves if successful auth doesn't complete in a minute or two
    } else {
      // create UserObject entity, register with Nexus on our same execution context,
      // pass address to callback
    }
  }
}

public class LoginManager implements Singleton, LoginService {
  public LoginManager (Nexus nexus) {
    nexus.registerKeyedFactory(UserManager.class, new KeyedFactory<UserManager>() {
      public UserManager create (Nexus nexus, Comparable<?> key) {
        return new UserManager(nexus, (String)key);
      }
    });
  }

  // from LoginService
  public void authenticate (String username, final String passHash,
                            final Callback<Address<UserObject>> callback) {
    // process the authentication on the user manager's execution context
    nexus.invoke(UserManager.class, username, new Action<UserManager,User>() {
      public void invoke (UserManager userMgr) {
        return userMgr.authenticate(passHash, callback);
      }
    });
  }
}

Threading Guarantees

It is important to note that the only enforcement of the single-thread-at-a-time nature of an entity's execution context is if all calls to an entity's methods are wrapped in actions and dispatched through a Nexus. The easiest way to avoid accidentally calling an entity's methods directly is to avoid making references to entities available. If the only want to get a reference to an entity is to call Nexus.invoke then you can have confidence that a programmer will not accidentally call directly into an entity's methods from the wrong thread.

Care must be taken when a reference to an entity is "ambiently visible" due to the automatic capture of the this pointer by anonymous inner classes. The following example demonstrates one way to mistakenly make use of that reference:

public class UserManager implements Singleton {
  public String getEmail (int userId) { ... }
}

public class MailManager implements Singleton {
  private Nexus nexus = ...;

  public void sendMail (final int userId, final String subj, final String body) {
    // BAD: this is the wrong way to structure this interaction
    nexus.invoke(UserManager.class, new Action<UserManager>() {
      public void invoke (UserManager userMgr) {
        String addr = userMgr.getEmail(userId);
        // BAD BAD BAD: we're running on the UserManager context but calling a method on the
        // MailManager through the this pointer captured by this anonymous inner class
        sendMail(addr, subj, body);
      }
    });

    // GOOD: this is the right way to structure this interaction
    String addr = nexus.request(UserManager.class, new Request<UserManager>() {
      public String invoke (UserManager userMgr) {
        return userMgr.getEmail(userId);
      }
    }
    sendMail(addr, subj, body);
  }

  protected void sendMail (String addr, String subj, String body) { ... }
}

Nexus provides a runtime safety check that helps to catch errors like this by nulling out the captured this pointers of Action and Request classes. This check is unfortunately too expensive to enable in production code, but one can (and should!) enable it during development by setting the nexus.saftey_checks system property to true on their server VM. When this safety check is enabled, code like the above will fail with a NullPointerException when it tries to call sendMail from inside the Action running in the UserManager's execution context.