querying devices by static attributes #835

butteronarchbtw · 2024-06-07T08:55:19Z

IoT Agent JSON version the issue has been seen with

everything above 1.26.0

Bound or port used (API interaction)

Northbound (Provision API and NGSI Interactions)

NGSI version

NGSIv2

Are you running a container?

Yes, I am using a contaner (Docker, Kubernetes...)

Image type

normal

Expected behaviour you didn't see

After provisioning a device and service I expected them to be created as a context broker entity (as stated in the FIWARE tutorial) in order to query them by static attributes like other entities e.g. ?q=owner=="User1" or ?type=Sensor. This behaviour is crucial for my application and I found no documentation where this feature has gone since version 1.26.0. I saw something similar mentioned in #832 but for this specific feature there should exist an alternative.

Unexpected behaviour you saw

Only measurements I send to the created device are listed as entities. When querying for a specific device or type that is not a type of a measurement, [] is returned.

Steps to reproduce the problem

Create a service.

curl -iX POST \
  'http://localhost:4041/iot/services' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "services": [
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "http://orion:1026",
     "entity_type": "Thing",
     "resource":    "/iot/d"
   }
 ]
}'

Create a device.

curl -iX POST \
  'http://localhost:4041/iot/devices' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
   {
     "device_id":   "motion001",
     "entity_name": "urn:ngsi-ld:Motion:001",
     "entity_type": "Motion",
     "timezone":    "Europe/Berlin",
     "attributes": [
       { "object_id": "c", "name": "count", "type": "Integer" }
     ],
     "static_attributes": [
       { "name":"refStore", "type": "Relationship", "value": "urn:ngsi-ld:Store:001"}
     ]
   }
 ]
}
'

Do a query for the device, e.g. request the specific device entity

curl -G -X GET \
  'http://localhost:1026/v2/entities/urn:ngsi-ld:Motion:001' \
  -d 'type=Motion' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /'

Configs

environment:
  - "IOTA_CB_HOST=orion"
  - "IOTA_CB_PORT=1026"
  - "IOTA_NORTH_PORT=4041"
  - "IOTA_REGISTRY_TYPE=mongodb"
  - "IOTA_MONGO_HOST=mongodb"
  - "IOTA_MONGO_PORT=27017"
  - "IOTA_MONGO_DB=iotagent-json"
  - "IOTA_HTTP_PORT=7896"
  - "IOTA_PROVIDER_URL=http://iot-agent:4041"

Log output

No response

The text was updated successfully, but these errors were encountered:

fgalan · 2024-06-11T12:03:51Z

After provisioning a device and service I expected them to be created as a context broker entity (as stated in the FIWARE tutorial) in order to query them by static attributes like other entities e.g. ?q=owner=="User1" or ?type=Sensor. This behaviour is crucial for my application and I found no documentation where this feature has gone since version 1.26.0. I saw something similar mentioned in #832 but for this specific feature there should exist an alternative.

The behaviour changed in IOTA-JSON 2.3.0, as explained in this comment on the #832 issue you cite.

The entity associated to a given device is not created on device provisioning, but when the first device measure is received. Static attributes are included in that moment.

If you need static attributes in advance, you can pre-create the entity at CB using the NGSIv2 API, including the static attributes in that request.

(Removing "bug" label as this is not a bug but expected behaviour)

fgalan · 2024-06-11T12:09:30Z

With regards to documentation https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/api.md#config-groups

Once a measure is received by the IoT Agent, the apikey and resource are used to identify the config group to which the device belongs. The config group is used to map the measure to a particular entity type and to provide the information needed to interact with the Context Broker.

If the device already exists in the Context Broker, the IoT Agent will update the entity with the new values. If the device does not exist, the IoT Agent will create it with the information provided by the config group and eventually will also create the entity in the Context Broker. This last operation is only possible if the IoT Agent is configured to use autoprovisioning.

https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/api.md#devices

If devices are not pre-registered, they will be automatically created when a measure arrives to the IoT Agent - this process is known as autoprovisioning. The IoT Agent will create an empty device with the group apikey and type - the associated document created in database doesn't include config group parameters (in particular, timestamp, explicitAttrs, active or attributes, static and lazy attributes and commands). The IoT Agent will also create the entity in the Context Broker if it does not exist yet.

butteronarchbtw · 2024-06-12T09:38:57Z

Thanks for the thorough response!

The following part still doesn't not work for me.

The entity associated to a given device is not created on device provisioning, but when the first device measure is received. Static attributes are included in that moment.

As you can see when provisioning the device as follows,

curl -iX POST \
  'http://localhost:4041/iot/devices' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
   {
     "device_id":   "motion001",
     "entity_name": "urn:ngsi-ld:Motion:001",
     "entity_type": "Motion",
     "timezone":    "Europe/Berlin",
     "attributes": [
       { "object_id": "c", "name": "count", "type": "Integer" }
     ],
     "static_attributes": [
       { "name":"refStore", "type": "Relationship", "value": "urn:ngsi-ld:Store:001"}
     ]
   }
 ]
}
'

the static attribute refStore gets added to that device.

But when querying the CB for the device after adding a measurement as you described, it returns the following.

[
	{
		"id": "Thing:motion001",
		"type": "Thing",
		"TimeInstant": {
			"type": "DateTime",
			"value": "2024-06-12T09:28:18.836Z",
			"metadata": {}
		},
		"c": {
			"type": "Text",
			"value": "1",
			"metadata": {
				"TimeInstant": {
					"type": "DateTime",
					"value": "2024-06-12T09:28:18.836Z"
				}
			}
		}
	}
]

Which does not include the refStore static attribute contrary to what you stated.

Is it me doing something wrong there?

fgalan · 2024-06-12T11:19:16Z

~~As far as I understand you have tested using ITOA-JSON 1.26.0 (please tell me otherwise).~~ Unsure...

I'd suggest to test with the newest IOTA-JSON version at the time of writing this (3.5.0) and tell us how it goes [Edit: or confirm that your test was done with that version]

AlvaroVega · 2024-06-12T11:21:31Z

Dear @butteronarchbtw

Creating a device (iotagentjson):

curl -iX POST   'http://localhost:4052/iot/devices'   -H 'Content-Type: application/json'   -H 'fiware-service: smartcity'   -H 'fiware-servicepath: /'   -d '{        
 "devices": [
   {                                                 
     "device_id":   "motion005", "apikey": "APIKEY1",
     "entity_name": "urn:ngsi-ld:Motion:001",
     "entity_type": "Motion",
     "timezone":    "Europe/Berlin",
     "attributes": [                                           
       { "object_id": "c", "name": "count", "type": "Integer" }
     ],                    
     "static_attributes": [                                                            { "name":"refStore", "type": "Relationship", "value": "urn:ngsi-ld:Store:001"}
     ]
   }
 ]
}
'

Sending a measure:

 curl -i -X POST 'http://localhost:7897/iot/json?i=motion005&k=APIKEY1' -d '{ "level": "33"}' -H 'content-type: application/json'

and querying to CB:

curl -i -X GET 'http://localhost:10026/v2/entities?type=Motion' -H 'fiware-service: smartcity'

The response has the static attribute:

[{"id":"urn:ngsi-ld:Motion:001","type":"Motion","TimeInstant":{"type":"DateTime","value":"2024-06-12T11:16:29.905Z","metadata":{}},"level":{"type":"Text","value":"33","metadata":{"TimeInstant":{"type":"DateTime","value":"2024-06-12T11:16:29.905Z"}}},"refStore":{"type":"Relationship","value":"urn:ngsi-ld:Store:001","metadata":{"TimeInstant":{"type":"DateTime","value":"2024-06-12T11:16:29.905Z"}}}}

butteronarchbtw · 2024-06-12T12:56:04Z

Sorry, I forgot to specify what version I was using. Beforehand I was using version 3.3.0.

Now I've upgraded to the most recent version, being 3.5.0. I can confirm this by the following:

curl -X GET 'http://localhost:4041/iot/about'

Which yields me {"libVersion":"4.5.0","port":"4041","baseRoot":"/","version":"3.5.0"}, which should be right.

But I can still not confirm that static attributes are returned by the query, also with the commands @AlvaroVega posted.

The exact steps I took to reproduce my issue on version 3.5.0 were:

Creating a service

curl -iX POST \
  'http://localhost:4041/iot/services' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "services": [
   {
     "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
     "cbroker":     "http://orion:1026",
     "entity_type": "Thing",
     "resource":    "/iot/json"
   }
 ]
}'

Creating a device - note that refStore is a static attribute

curl -iX POST \
  'http://localhost:4041/iot/devices' \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: openiot' \
  -H 'fiware-servicepath: /' \
  -d '{
 "devices": [
   {
     "device_id":   "motion001",
     "entity_name": "urn:ngsi-ld:Motion:001",
     "entity_type": "Motion",
     "timezone":    "Europe/Berlin",
     "attributes": [
       { "object_id": "c", "name": "count", "type": "Integer" }
     ],
     "static_attributes": [
       { "name":"refStore", "type": "Relationship", "value": "urn:ngsi-ld:Store:001"}
     ]
   }
 ]
}
'

Sending a measurement

curl -iX POST \
  'http://localhost:7896/iot/json?k=4jggokgpepnvsb2uv4s40d59ov&i=motion001' \
  -H 'Content-Type: application/json' \
  -d '{"c": "1"}'

Querying the CB for the device

curl -X GET 'http://localhost:1026/v2/entities' -H 'fiware-service: openiot' -H 'fiware-servicepath: /'

Which leaves me with

[
  {
    "id": "Thing:motion001",
    "type": "Thing",
    "TimeInstant": {
      "type": "DateTime",
      "value": "2024-06-12T12:47:21.741Z",
      "metadata": {}
    },
    "c": {
      "type": "Text",
      "value": "1",
      "metadata": {
        "TimeInstant": {
          "type": "DateTime",
          "value": "2024-06-12T12:47:21.741Z"
        }
      }
    }
  }
]

The response still does not contain refStore.

AlvaroVega · 2024-06-12T13:18:16Z

@butteronarchbtw Why don't you include apikey in device at provisioning time? Otherwise only service (group) info about static attributes will be provisioned (since even entity_type is not the same for device and group)

In your case 2 devices are finally provisioned.

butteronarchbtw · 2024-06-12T13:28:52Z

@AlvaroVega That was it! Either using the same entity_type for the device and service and/or using the apikey during provisioning solves the issue. Thank you!

But since these commands were taken straight from the docs https://fiware-tutorials.readthedocs.io/en/latest/iot-agent-json.html, these should be updated I think. Is there a way I could contribute to this or are these handled internally?

fgalan · 2024-06-12T13:36:29Z

But since these commands were taken straight from the docs https://fiware-tutorials.readthedocs.io/en/latest/iot-agent-json.html, these should be updated I think. Is there a way I could contribute to this or are these handled internally?

I think the maintainer of the tutorial documenation is @jason-fox . I understand he can tell you how to contribute to it.

In addition to that, if you find that the documentation of the iotagent-node-lib repository should be also improved based on your experience, please feel free to do a pull request with the contribution. We would be more than happy to review it and improve the IOTAs documentation :)

Apart from that, I understand this issue is solved and can be closed.

butteronarchbtw added the bug label Jun 7, 2024

github-actions bot added the NGSIv2 label Jun 7, 2024

fgalan removed the bug label Jun 11, 2024

fgalan closed this as completed Jun 12, 2024

butteronarchbtw mentioned this issue Jun 12, 2024

fix issue with static attributes in the current tutorial requests FIWARE/tutorials.IoT-Agent-JSON#11

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

querying devices by static attributes #835

querying devices by static attributes #835

butteronarchbtw commented Jun 7, 2024

fgalan commented Jun 11, 2024

fgalan commented Jun 11, 2024

butteronarchbtw commented Jun 12, 2024 •

edited

Loading

fgalan commented Jun 12, 2024 •

edited

Loading

AlvaroVega commented Jun 12, 2024 •

edited

Loading

butteronarchbtw commented Jun 12, 2024

AlvaroVega commented Jun 12, 2024 •

edited

Loading

butteronarchbtw commented Jun 12, 2024 •

edited

Loading

fgalan commented Jun 12, 2024

querying devices by static attributes #835

querying devices by static attributes #835

Comments

butteronarchbtw commented Jun 7, 2024

IoT Agent JSON version the issue has been seen with

Bound or port used (API interaction)

NGSI version

Are you running a container?

Image type

Expected behaviour you didn't see

Unexpected behaviour you saw

Steps to reproduce the problem

Configs

Log output

fgalan commented Jun 11, 2024

fgalan commented Jun 11, 2024

butteronarchbtw commented Jun 12, 2024 • edited Loading

fgalan commented Jun 12, 2024 • edited Loading

AlvaroVega commented Jun 12, 2024 • edited Loading

butteronarchbtw commented Jun 12, 2024

AlvaroVega commented Jun 12, 2024 • edited Loading

butteronarchbtw commented Jun 12, 2024 • edited Loading

fgalan commented Jun 12, 2024

butteronarchbtw commented Jun 12, 2024 •

edited

Loading

fgalan commented Jun 12, 2024 •

edited

Loading

AlvaroVega commented Jun 12, 2024 •

edited

Loading

AlvaroVega commented Jun 12, 2024 •

edited

Loading

butteronarchbtw commented Jun 12, 2024 •

edited

Loading