Skip to content

How To: Add a new command message in Arkouda

Jump to bottom
glitch edited this page Jan 28, 2022 · 5 revisions

PENDING Issue #1049, PR#1052

Goal

This article will walk through the basic steps of adding a new client <---> server command and message response to Arkouda. At its conclusion you should have a general idea of how to add new server functionality and wire its invocation into the python client.

Story

Let's start with a basic scenario where a CarEntry was recently added as a complex object managed by the server. A client can interact with instances of a CarEntry by passing command messages to the server. We'll assume various commands and procedures already exist for the CarEntry and our task will be implementing a new command to set the color of the car.

Getting Started

There are a number of files, classes, and procedures we need to understand to create our new Command. Let's outline a basic skeleton as a starting point.

Client side

In the python client-side code we'll assume we have a cars.py with a class Car in it. Our job is to add a new function to this class which will set the color of the car entry stored on the server. Let's call this new function set_color(color:str)

class Car:
  # ... initialization code etc.

  def set_color(color:str):
    # TODO: Implement the request to set the car color over on the server.
    pass

Server side

In the server source-code we generally have a module containing all of the procedures etc. related to the modification & operations we wish to apply to an object stored on the server. As the code for an object grows more complex we can break it into separate modules/files to aid in organization. However, for the purposes of our development story, we are going to use a single module named CarMsg in a file named CarMsg.chpl. We are going to assume this file was already created (along with corresponding pieces in other modules) when the original CarEntry code was implemented.

module CarMsg {

    // ... pre-existing code ...

    proc setCarColorMsg(cmd:string, payload:string, st:borrowed SymTab): MsgTuple throws {
        // This is the code we are going to add.
        // We'll discuss the message signature after we understand more of the pieces
    }
}

Server-side Wiring

Before we can finish implementing the code in our basic skeleton, let's take a look at various components of the server to understand how the parts are wired together. Once we understand how those parts work, we'll have a better idea of what we need to add to cars.py and CarMsg.chpl.

Component Overview

Creating a brand new Entry object is beyond the scope of this How-To article so we are going to assume a number of convenience functions and other pieces of connective code already exist. However, we need to have a general understand of what they are, briefly:

  • MultiTypeSymEntry.chpl - This is where the Symbol Entry definitions and types live for our complex server objects. For our story we are assuming a CarEntry already exists here.
  • MultiTypeSymbolTable.chpl - Contains the code for our Symbol Table (SymTab) which holds the references to our named objects in memory. Generally there are convenience functions for retrieving our entry by name/id and casting it to the proper type; we will assume this already exists as getCarEntry which will handle the look-up by id and return us a CarEntry object.
  • CarMsg.chpl - This is where our CarMsg module lives and where we plan to add our new setCarColorMsg.
  • Message.chpl - This contains the definition of our MsgTuple record which is the response object passed back to the client. We won't need to modify this, but it's important to know that it's the Msg wrapper of the CarMsg which contains the information we want to pass back to the client.
  • ServerModules.cfg - In our modular build process, this is where we configure which modules are included in the build. There is a behind the scenes process which happens at build time which reads this configuration file and generates some code responsible for registering commands with the server which will look at in more depth later. For now, we will assume our CarMsg module is already listed in this file.
  • ServerRegistration.chpl - This is an auto-generated file based on the ServerModules.cfg. We never modify this file directly, but we'll see how it helps pull the pieces together later.
  • CommandMap.chpl - This is the module which contains our cmd->function routing table. Ultimately this will be storing our setCarColorCmd binding once we understand how it gets populated.
  • arkouda_server.chpl - This is the module containing our main run-time loop which handles sending & receiving messages over a socket. When an arkouda_server is started this kicks off command/function registration, owns the command map, and contains the entry point to our command routing process. It ties all of the pieces together.

Now that we have an understanding of the various cast members on the server we need to gain an understanding of:

  1. Server compilation & flow of execution
  2. Command routing and processing

Server startup and Command registration

Our first flow of control is server start up which is how our command gets registered with the server; understanding how this happens will help us understand where/why we need to put our new code.

  1. When the arkouda_server starts it enters the main procedure and invokes the registerServerCommands() procedure.
  2. registerServerCommands() manually registers some of our base commands, but more importantly invokes the doRegister() function. We need to take a brief diversion here.
  • When we built/compiled the server code there was a dynamic code generation process which took place behind the scenes. A python script named serverModuleGen.py was run which reads the ServerModules.cfg file and generates a Chapel file named ServerRegistration.chpl
  • ServerRegistration.chpl is where the doRegister() procedure is defined and it is responsible for importing modules and invoking their registerMe() procedure.
  • The registerMe() procedure defined in each module is where the string command-name and corresponding function is added to the CommandMap which acts as a basic routing table.

Server command routing & processing

  1. In arkouda_server, once the registerServerCommands() procedure completes we enter the main server loop where our server awaits incoming messages over a ZeroMQ socket. For each message received we parse it for a cmd and payload
  2. For each command request received from the client, we parse the message for a command name cmd and arguments which we call the payload.
  3. With the cmd name parsed control enters the select cmd {} section where we route execution to the appropriate procedure based on the string literal cmd. For our purposes of the setCarColorCmd it should now reside in the CommandMap. After the server tests for a core set of commands, it then looks in the CommandMap for the dynamically registered commands and executes the corresponding function; in our case this will be setCarColorMsg.

Recap

Let's briefly recap what we know so far.

When we build the arkouda_server (via make) a behind the scenes process reads our ServerModules.cfg to see which modules we want to include in our build. This auto-generates a Chapel file named ServerRegistration.chpl imported by arkouda_server.chpl which invokes its sole function doRegister() function. This hook contains all of the import <module> and <module>.registerMe() calls which in-turn lets each module register a string-literal command name to a call back function in the CommandMap.

When the arkouda_server process is started, after all of the registration takes place, we enter our main server execution loop which starts listening on a ZeroMQ socket for incoming command messages. When a request message is received, the command is parsed along with its arguments and then looked up in the CommandMap to find the corresponding function responsible for executing the client's request.

In our story, the server will receive a message with the command setCarColorCmd and arguments containing our object's name and new color. setCarColorCmd will be looked up in the CommandMap which invokes our CarMsg.setCarColorMsg(...) function responsible for retrieving our CarEntry, setting its color, and generating the response of whether or not the operation was successful.

Function / Msg signature

Now that we have the general idea, let's take a quick look at the function signature so we can understand what the arguments are. Recall our skeleton message proc

module CarMsg {

    // ... pre-existing code ...

    proc setCarColorMsg(cmd:string, payload:string, st:borrowed SymTab): MsgTuple throws {
        // This is the code we are going to add.
        // We'll discuss the message signature after we understand more of the pieces
    }
}

Notice the function signature: proc setCarColorMsg(cmd:string, payload:string, st:borrowed SymTab): MsgTuple throws

Chapel requires all of the function signatures inside of the CommandMap to have a matching signature and return type. Regardless of whether or not we plan to use the arguments or look something up in the SymTab we need our Msg function to match. The structure of the payload is generally up to the developer, but you should look at some of the other implementations to see what gets passed in. In our story, we're going to need the id of the entry in the SymTab so we can retrieve it, along with a string-literal containing the value to store for the color. This can be a comma-delimited or json formatted string. As your messages become more complex we generally advise something like json but at current we haven't formally made this a requirement.

As for the return type, this will contain a string-literal that will be wrapped in a Messages.MsgTuple and passed back to the client. MsgTuple takes two arguments, a string-literal message and an enum MsgTyp which indicates to the receiver the type of the Msg NORMAL, WARNING, ERROR. You can also return a binary message, but that is beyond the scope of this How-To article. It's generally the same process but binary return messages are contained in a separate internal command map since they require slightly different return handling.

registerMe

The final piece of the puzzle is actually registering your message. We've seen the various pieces and hooks to understand how the parts are wired together. We'll need to add CarMsg to ServerModules.cfg. When ServerRegistration.chpl gets auto-generated it will contain the following at a minimum.

proc doRegister() {
  import CarMsg;
  CarMsg.registerMe();
}

This means CarMsg.registerMe() will be invoked on startup. Our next step is to add/update the registerMe() function in CarMsg where we add our actual cmd string to our function. Recall, we decided we're going to use the string setCarColorCmd and bind it to the function CarMsg.setCarColorMsg(...). Thus our registerMe() function in CarMsg will be along the lines of:

proc registerMe() {
    use CommandMap;
    registerFunction("setCarColorCmd", setCarColorMsg);
}

If you look at CommandMap.chpl you'll see the functions you can call.

Client

In the client we use generic_msg to handle sending the message to the server which takes two arguments: cmd and args. You can take a look at some of the other client implementations to see how it's done but in general:

def doOp(myarg:str):
  result = generic_msg(cmd="my_command", args="my, arg, string")

Exercise for the Reader

Now that we understand how the pieces work together, let's take another look at our skeleton along with the various snippets of code we'll need to add to the supporting pieces.

At this point you should attempt to add the code to the various pieces before moving on. Below we'll look at the general solution. NOTE: You don't have to add the logic to actually retrieve the CarEntry and update its color; it's fine to stub it out at this point as long as you understand what the arguments are.

... ... ...

Example Solution

  1. In ServerModules.cfg you should have added the line:
CarMsg
  1. In CarMsg.chpl you should have something like the following
module CarMsg {

    // ... pre-existing code ... and various imports

    proc setCarColorMsg(cmd:string, payload:string, st:borrowed SymTab): MsgTuple throws {
        try {
            // Parse payload for the id & color
            var (name, color) = payload.splitMsgToTuple(2);
            var car = getCarEntry(name, st); // expected convenience function to retrieve our car object
            car.setColor(color);
            return new MsgTuple("Success", MsgType.NORMAL);
        } catch {
            return new MsgTuple("Error when retrieving car and/or setting color", MsgType.ERROR);
        }
        
    }

    proc registerMe() {
        use CommandMap;
        registerFunction("setCarColorCmd", setCarColorMsg);
    }
}
  1. Client cars.py Car class should contain something like
class Car:
  # ... pre-existing code ...
  def set_color(color:str):
    cmd = "setCarColorCmd"
    args = {"name":self.name, "color":color} # note: this could be a csv or space delimited string, but we'll dump to json
    result = generic_msg(cmd=cmd, args=json.dumps(args))
    # parse the result

This concludes this How-To lesson. Hopefully you have a general understanding of how to add new messages and commands to the client & server.

Clone this wiki locally