Skip to content

2. Parsing KPN LoRa

Jump to bottom
timmehovinga edited this page Sep 16, 2016 · 13 revisions

In the first exercise you created your own server to listen and respond to HTTPS posts. In the second exercise we will move on to receive LoRa payload and parse it to be able to manipulate the data.

#Forwarding KPN LoRa to NodeRed Let's now point our data to our /lorapost listener we've created in the first exercise, if you haven't done already:

  • KPN LoRa Developer Portal: go to your dashboard, click on devices and edit the device you want to forward. Under the Destination URL fill in the destination URL from your first exercise, for instance 'https://yourname.eu-gb.mybluemix.net/lorapost'. If you now hit the test button, you will see a simulated KPN LoRa message in the debug tab of NodeRed.

  • KPN Thingpark (contracted customers only): You will need to go to your deviceManager and adjust the application profile to have a destination URL matching the destination from the first exercise. Important here is to add an extra '/' at the end for it to work (so https://yourname.eu-gb.mybluemix.net/lorapost/ for example).

  • You will see a lot of information coming into your application server. To easily inspect all elements of the message we have created a github repository showing a example of a LoRa message here. There is a slight difference between using the KPN Developer Portal or KPN Thingpark (contracted customers only):

    1. A KPN LoRa post consists of information in the query-parameters, some headers (which we are not going to use for now) and a J-SON body. Note that you have to select the J-SON output in Thingpark Application Server to get J-SON instead of XML.
    2. The J-SON body has a payload_hex parameter or, in case of Thingpark, it has a extra layer DevEUI_uplink which again contains a payload_hex. Can you spot other parameters like the Spreading Factor (SF) or DevEUI?
    3. The query-parameters include a AS_ID and a Token. The token was generated by the server with a SHA256 method using a secret SHA key. Later, in the next exercise, we will check the validity of this key to know for sure that the message originates from our own device.

ProTip: To speed up your development you don't want to wait every 5 minutes for a LoRa message to check if a change works on your machine. Plus you might want to check what response your server gives. The KPN LoRa Developer portal has a test-payload functionality for this. If you edit a device you can click on test to simulate a LoRa payload from that device. If you want to have more control over the message we have created simple simulator instructions here, using Postman to post message equivalent to a LoRa message as much as you like!

#Unpacking LoRa Data Let's now pull out some relevant data from the LoRa messages:

  • Go to your NodeRed flow editor and add a new function node. Call it "Parse Catch Input". In this function node we select our LoRa payload directly through msg.payload.payload_hex (KPN Developer Portal) or through msg.payload.DevEUI_uplink.payload_hex (KPN Thingpark. We save the right body of the message to a separate body parameter so we can use the same script for both KPN developer portal and Thingpark from now on.
//Uncomment if you are using Thingpark
//body = msg.payload.DevEUI_uplink;

//Uncomment if you are using Developer Portal 
body = msg.payload;

msg.lorapayload = body.payload_hex;
msg.body = body;

return msg;

  • Hookup a debug node, and adjust it's output to show msg.lorapayload. Post a LoRa message and see the result: we have now easy access to our LoRa message.

Creating Errors

We are not there yet, however. To make a robust application server we need to think ahead a bit and make sure we do some checks first on the format of the message. If a message does not have a payload_hex parameter it will generate an error here. That one is probably easy to spot, but if you want to use a parameter like DevEUI later on it might take a lot of your debugging time to find out that this one was not in the message.

  • Let's check for a correct payload. We will do that using the hasOwnProperty(parameter) function. Which gives a true if a object has that parameter and false if not. Also, we will generate an error using the node.error(errormessage, msg) function of NodeRed. While we're at it, we have also included a automatic check if we are receiving message from the developer portal or Thingpark directly. Adjust your parsing function node to this code:
// Check if the body has a DevEUI_uplink parameter (Thingpark) or uses a simple format (KPN LoRa Developer Portal)
if (msg.payload.hasOwnProperty("DevEUI_uplink")) {
    body = msg.payload.DevEUI_uplink;
} // Check if the body has the right parameters for Developer Portal
else if (msg.payload.hasOwnProperty("payload_hex")){
    // Developer portal has a simpler json format without DevEUI_uplink
    body = msg.payload;
}else{
    node.error("Not a valid Thingpark or Developer Portal message", msg);
    return;   
}

msg.lorapayload = body.payload_hex;
msg.body = body;

return msg;

Note that it is very important to include an empty return after generating an error to stop the flow moving forward!

  • Let's check how this works. Go to your postman and create a post to your /lorapost that has a body containing random data (but not J-SON formatted body :)). If you now hit sent you will see a error in the debugger.

.

  • Now we know in our NodeRed flow that an error has occurred. The eagle-eyed observer might have noticed that your postman still received a message like everything was fine. Let's help our future selves and generate an error there as well. We will need to adjust the message to our HTTP Response Node.
  • Under input nodes you can find a node titled "catch". Drag and drop this to your flow. This nodes catches errors generated in your flow. You can select which nodes to catch errors from, for now the default setting of all nodes is fine.
  • Add a function node and connect its input to the output of your catch node. Name it "Handle Error". Use the below code to add some information to the error:
// Add a topic property to our message to let other nodes know
// that this message is an error.
msg.topic = "error";

// The payload of this error message should say something 
// about the error. This payload is going to be passed back
// to our HTTPS Response node. Note that the msg.error.message
// is the message we gave as input in the node.error()
msg.payload = "Encoutered Error: "+ msg.error.message;

return msg;
  • Now connect the output of your "Handle Error" node to the "Create Response" node. Also make sure that the output of "Parse Catch Input" is connected to your response to get the flow in the below picture:

  • Open your "Create Response" function and adjust to see if we deal with an error or a valid message for the response
if (msg.topic === "error"){
    // In case of an error, do nothing with the message and
    // pass it on to the reponse node. The msg.payload
    // already contains the right information
    return msg;
}else{
    // We know we have received a valid message
    // Let's just pass back the lora data we have received
    msg.payload = "Received payload: " + msg.lorapayload;
}
return msg;
  • Now check if everything works. Create a random post from postman and see both the NodeRed debugger and the reponse to postman output a nice error message. Try a LoRa post (real, simulated by postman or through the test functionality in the developer portal) and see the result in your debugger and as response. Does it work? Great :)!
  • Almost there for this exercise. Now we have set up this mechanism, let's checks the query-parameters and a few more body-parameters in the parser to make sure we have everything we need in the LoRa message for the rest of the flow. Go to your "Parse Catch Input" node to implement this code:
// Check if the body has a DevEUI_uplink parameter (Thingpark) or uses a simple format (KPN LoRa Developer Portal)
if (msg.payload.hasOwnProperty("DevEUI_uplink")) {
    body = msg.payload.DevEUI_uplink;
} // Check if the body has the right parameters for Developer Portal
else if (msg.payload.hasOwnProperty("payload_hex")){
    // Developer portal has a simpler json format without DevEUI_uplink
    body = msg.payload;
}else{
    node.error("Not a valid Thingpark or Developer Portal message", msg);
    return;   
}

// Check if the query paramaters are there before continuing
var RequiredKeys = ["LrnDevEui","LrnFPort","LrnInfos","AS_ID","Time","Token"];
for(var i=0; i < RequiredKeys.length; i++) {
    key = RequiredKeys[i];
    if (!msg.req.query.hasOwnProperty(key)){
        node.error("Missing Query Parameter '" + key + "'", msg);
        return;
    }
}

// Check if the body has the right parameters
RequiredBodyElements = ["CustomerID","DevEUI","FPort","FCntUp","payload_hex"];
for(var i=0; i < RequiredBodyElements.length; i++) {
    key = RequiredBodyElements[i];
    if (!body.hasOwnProperty(key)){
        node.error("Missing DevEUI_uplink Element  '" + key + "'", msg);
        return;
    }
}

msg.lorapayload = body.payload_hex;
msg.body = body;

return msg;
  • Read the code above and try to figure out how we walk to our list of requirement elements, do a check for every element and generate an error if we miss something.

Exciting! We have neatly parsed our LoRa payload! Now it's time to move to the next exercise and see if we are not receiving fake LoRa messages by checking our token.

You can find the complete implementation of the exercise at https://github.com/iotacademy/NodeRed_KPN_LoRa/blob/master/02_ParsingKPNLoRa.md if you are stuck somewhere.

Clone this wiki locally