Skip to content

How does the dashboard visualization connect to Entity telemetry data?

Jump to bottom
liyuanqian edited this page Jul 14, 2023 · 2 revisions

Telemetry data in AWS IoT TwinMaker is received through the GetPropertyValueHistory API. In this demo we're invoking this API using entity requests (for details on different kinds of AWS IoT TwinMaker data connectors please see our documentation [1]). For entity requests, we query for specific telemetry data as identified by its (entity, component name, property name) reference.

Physical assets are modeled in AWS IoT TwinMaker as entities, and their details (such as static properties, relationships with other entities, and time-series telemetry data properties) are modeled in components that are then attached to the entity. For deeper discussion of this topic see this HowTo [1]. Let's take a closer look at the entity model for the Freezer Tunnel [2]:

...
{
  "components": {
    "CookieLineComponent": {
      "componentTypeId": "com.example.cookiefactory.cookieline",
      "properties": {
        "alarm_key": {
          ...
        },
        "telemetryAssetId": {
          ...
        },
        "telemetryAssetType": {
          ...
        }
      }
    },
    "EquipmentComponent": {
      "componentTypeId": "com.example.cookiefactory.equipment",
      "properties": {
        "belongTo": {
          ...
        },
        "feed": {
          ...
        }
      }
    }
  },
  "description": "",
  "entityId": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298",
  "entityName": "FREEZER_TUNNEL",
  "parentEntityId": "COOKIE_LINE_5ce9f1d5-61b0-433f-a850-53fa7ca27aa1"
},
...

We can see that it has 2 components: CookieLineComponent as defined by the componentTypeId com.example.cookiefactory.cookieline and EquipmentComponent as defined by com.example.cookiefactory.equipment. Let's take a closer look at the cookieline component type definition [1]

{
  "componentTypeId": "com.example.cookiefactory.cookieline",
  ...
  "functions": {
    "dataReader": {
      "implementedBy": {
        "isNative": false,
        "lambda": {
          "arn": "__TO_FILL_IN_SYNTHETIC_DATA_ARN__"
        }
      },
      "isInherited": true
    }
  },
  ...
  "propertyDefinitions": {
    ...
    "Speed": {
      "dataType": {
        "type": "DOUBLE"
      },
      "isExternalId": false,
      "isFinal": false,
      "isImported": false,
      "isInherited": false,
      "isRequiredInEntity": false,
      "isStoredExternally": true,
      "isTimeSeries": true
    },
    "Temperature": {
      "dataType": {
        "type": "DOUBLE"
      },
      "isExternalId": false,
      "isFinal": false,
      "isImported": false,
      "isInherited": false,
      "isRequiredInEntity": false,
      "isStoredExternally": true,
      "isTimeSeries": true
    },
    ...
  }
}

There are a number of fields defined in the model but for this discussion we'll focus on our data connectivity to the Temperature data. This is defined as a property of the component type [1]. We also define a dataReader element which specifies the Lambda ARN to invoke to retrieve data for any entity with this component type attached to it.

With these entity model details (FreezerTunnel entity with CookieLine component with Temperature property and dataReader definition) we have the minimal entity model elements to connect to data. When the GetPropertyValueHistory API is invoked to fetch Temperature data for the FreezerTunnel, it will send a request to the Lambda with a request payload and expects the Lambda to return back a response payload containing the requested Temperature data for a given time range.

Let's take a closer look at the sample Lambda used in this demo.

Synthetic Data Lambda example

The Lambda used in this demo returns synthetic data back to the application but can be modified to fetch data from any datasource accessible by AWS Lambda. (e.g. for an example that connects to Timestream see: [1]). Most of the request/response payload manipulation is handled by the udq_utils helper library. For details on using this library to implement GetPropertyValueHistory (also sometimes referred to as "Unified Data Query (UDQ)") see [2].

In summary, it consists of 3 main elements:

  1. Sample data json - a sample static JSON data file whose contents are repeatedly returned from UDQ to allow us to easily control what data is used in the demo [1].
    • The Lambda has some options for configuring what data file is used (e.g. can be used to dynamically update the Lambda function to change what data is returned) but by default it will use the demoTelemetryData.json file [1] [2]. We read this JSON data into a Pandas Dataframe [3] and include some sample code for common data preparation steps such as renaming ids and columns. [4]
  2. Implementation of UDQ using udq_utils - the sample implements the AWS IoT TwinMaker UDQ interface using udq_utils [1] for request/response payload handling [2]
    • This allows our code to focus on constructing IoTTwinMakerDataRow objects and delegates the JSON response construction to the library. A IoTTwinMakerDataRow object needs to support returning 3 parts to AWS IoT TwinMaker: 1/ the entity-component-property reference that identifies the data, 2/ the timestamp of the data row, 3/ the value of the data row. In this sample, we compute the value of the data row based on the value of the requested column (i.e. the "selected property") in the dataframe [1] [2]
  3. Data extrapolation logic - since the requested time-range will likely fall outside of the sample data, we extrapolate from the sample data to synthetically generate enough data points for the request [1].

With the above elements in mind, let's walk through a sample request to the Lambda:

When the GetPropertyValueHistory API is invoked with this AWS CLI command:

aws iottwinmaker get-property-value-history --region us-east-1 \
    --cli-input-json '{
  "workspaceId": "CookieFactoryV2",
  "entityId": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298",
  "componentName": "CookieLineComponent",
  "selectedProperties": [
    "Temperature"
  ],
  "startTime": "2023-07-11T01:00:00Z",
  "endTime": "2023-07-11T02:00:00Z",
  "orderByTime": "ASCENDING",
  "maxResults": 3
}'

The Lambda receives this event payload:

{
  "workspaceId": "CookieFactoryV2",
  "selectedProperties": [
    "Temperature"
  ],
  "startDateTime": 1689037200,
  "startTime": "2023-07-11T01:00:00Z",
  "endDateTime": 1689040800,
  "endTime": "2023-07-11T02:00:00Z",
  "properties": {
    "AlarmMessage": { "definition": { ... } },
    "Speed": { "definition": { ... } },
    "Temperature": { "definition": { ... } },
    ...
    "telemetryAssetId": { ..., "value": {"stringValue": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298"}},
    "telemetryAssetType": { ..., "value": {"stringValue": "CookieLine"}},
    ...
  },
  "entityId": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298",
  "componentName": "CookieLineComponent",
  "maxResults": 3,
  "orderByTime": "ASCENDING"
}

After the payload is processed by udq_utils the entity_query implementation hook is invoked and we use our _get_data_rows() implementation to handle the request. Based on the selected_properties in the request, we retrieve the relevant column from our pre-computed pandas dataframe and based on the request start and end times we calculate the offset into our sample telemetry data to start extrapolating data from. We iterate over the sample data repeatedly until we've generated enough response data and return the result as a list of RenderIoTTwinMakerDataRow objects. The udq_utils library takes this data and appropriately transforms / groups it under entity-component-property references to constructs the final JSON response payload to return from the Lambda:

{
  "propertyValues": [
    {
      "entityPropertyReference": {
        "entityId": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298",
        "componentName": "CookieLineComponent",
        "propertyName": "Temperature"
      },
      "values": [
        {
          "time": "2023-07-11T01:00:00.000000Z",
          "value": {
            "integerValue": "-18"
          }
        },
        {
          "time": "2023-07-11T01:00:10.000000Z",
          "value": {
            "integerValue": "-29"
          }
        },
        {
          "time": "2023-07-11T01:00:20.000000Z",
          "value": {
            "integerValue": "-15"
          }
        }
      ]
    }
  ],
  "nextToken": null
}

We then get the following result from AWS IoT TwinMaker to our initial AWS CLI GetPropertyValueHistory API request:

{
  "propertyValues": [
    {
      "entityPropertyReference": {
        "componentName": "CookieLineComponent",
        "externalIdProperty": {
          "alarm_key": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298",
          "telemetryAssetId": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298"
        },
        "entityId": "FREEZER_TUNNEL_e12e0733-f5df-4604-8f10-417f49e6d298",
        "propertyName": "Temperature"
      },
      "values": [
        {
          "value": {
            "integerValue": -18
          },
          "time": "2023-07-11T01:00:00Z"
        },
        {
          "value": {
            "integerValue": -29
          },
          "time": "2023-07-11T01:00:10Z"
        },
        {
          "value": {
            "integerValue": -15
          },
          "time": "2023-07-11T01:00:20Z"
        }
      ]
    }
  ]
}
Clone this wiki locally