[In progress] Implement all mandatory devices / clusters

[mfucci]

The goal is to provide a minimal implementation for all the mandatory devices / clusters as listed here.

Cluster status / contributors:

• MA-rootdevice [In progress]: @mfucci, @Apollon77

  • Basic Information Cluster [Specs completed, implementation in progress]
  • Descriptor Cluster [Specs in progress]
  • General Commissioning Cluster [Specs in progress]
  • Operational Commissioning Cluster [Specs in progress]
  • Others ?

• MA-onofflight [In progress]: @mfucci, @Apollon77

  • OnOff Cluster [Specs in progress]

• MA-aggregator [In progress]: @Apollon77

• MA-bridgeddevice [In progress]: @Apollon77

[Apollon77]

I would plan to do "MA-aggregator" and "MA-bridgeddevice" to basically add bridge support and the needed clusters for that ... then maybe more ... lets see soon

[Apollon77]

@mfucci if ok I would start with updating the existing clusters and implemnenting two/three more ... Then you can review if I understood the "xml to typescript" approach right and then I do the rest in smaller chunks. Or what do you would like?

Also: One PR per cluster or can I do it in chunks? What would be maximum stuff per RP from your point of view?

[mfucci]

Added you as a contributor for "MA-aggregator" and "MA-bridgeddevice" :-)

Fine to split as granular or as big as you want for the PR. You can also send me draft PR to review before they are completed if you want a feedback if you are going in the right direction.

"xml to typescript" conversion for cluster definition: let me know if you need any help. It will probably be worth to write a doc on how to do this since there is a lot of work to do on adding clusters and this will be helpful to have more contributors (you can start something and I can add in it).

I am using advanced typescript dark magic to auto-generate strong typing and be able to do parsing / unparsing of data structure. To be honest , this is even a little past my level of expertise and I am not sure exactly how it works and run into missing features in the Typescript languages. You probably don't need to go into the detail of how it works to add clusters but if you are a Typescript expert, feel free to suggest how to improve this logic. Or you can ping me to implement extra features if needed.

Data serialization / deserialization: probably everything needed is supported. Some checks are not implemented, like for instance making sure a bytestring as a length of 16.

Cluster definition: 80% there, we probably will need to add read-only flags and access control. Some features are completely missing like client-side command / attributes, hopefully you won't need them.

Device definition: this is not implemented at all yet. Since devices are defined in Matter, we probably want a matching definition in node-matter. As you can see in the code, I use clusters directly instead of using the device that encapsulate those clusters. Probably not mandatory to start implementing on your side but there will be a refactoring introducing device definitions at some point (you can start this if you want).

Thanks!

• Marco

[Apollon77]

I'm currently updating the already defined Clusters ... On one hand to understand how you designed it and on otther hand as practice ;-)

In fact I already stumbled over one topic where I need "dark type magic support" ... it is for:

• defaultValues for attributes .. i did something here, but in fact as soon as it has a defaultValue it is also kind of optional
• "optional" attributes ... here I did not came to any solution :-)

I will finish my work and then open a draft PR. maybe it make sense to iterate then on this together ;-)

xml2typeScript: In fact I started noting done some parts for me already, Idea was to add a "HOWTO_CLUSTER.md" somewhere with "how to convert stuff".

Device definition might not really be needed because this is defined (at least in my case) by other structuring-options, so rest is "dynamic" ... but we will see ... one step after the other :-)

[Apollon77]

@mfucci In order to continue here I would need something on the "Cluster" instance. As defined by Core spec 7.13 cluster should/must have some global Elements. I think it makes sense to dd this as additional struct because it is the same for all clusters. Again some fileds mandatory, some optional.

I would also propose to build the AttributeList automatically from the provided attributes, same pot for AcceptedCommandList ... or not ;-) In fact this would not start the discussion about how we implement Cluster logic. I propose to do this on example of the OnOff Cluster ...

[mfucci]

If I understand well 7.13, those are attributes present on every cluster.

I suggest to hard code them similar to what I did with the DescriptorCluster in InteractionServer#addEndPoint since the DescriptorCluster is present on every endpoint.

In Cluster.ts, the global element definitions can be added and automatically added to any Cluster definitions in the Cluster function.

On the controller-side: no need to do anything special since they will appear as normal attributes

On the server-side: their values (which is static value as far as I understand) can be initialized in ClusterServer. Some trick should be done on attributesInitialValues type so an initial values for those attributes don't need to be set (just using Omit<> should work).

[Apollon77]

Ok, lets try to kep it static based on our cluster implementation ... we will see if we find cases where this do not work. :-)
Should we add it together with OnOffCluster work/PR (because will be the first case to utilize it) or separately?

[mfucci]

Let's add it separately, so it will be easier to review / merge

[Apollon77]

For OnOffCluster i need something more. When implementing the OnOffClusterServer i need a constructor style code because I need to register atterListeners on two attributes to implement logic as soon as such a value changes. How I should do that? Right now I can only add implememtation for the defined commands, but nothing more on the "instance"

[mfucci]

Hmmm...

One way can be to have optional on[AttributeName]Update properties on the handler object. If they are used, the InteractionServer will automatically add them as listeners on the attribute.

Another way can be to have an "initialization" property taking the attributes as an arguments.

A third way would be to refactor the whole cluster server to use class instead of objects.

I would prefer the option 1 since it keeps the handler stateless and on[AttributeName]Update might give the chance to refuse an update if it doesn't satisfies some checks.

[Apollon77]

Yes with this the current requirements for OnOff would be fullfilled because I only "need" subscriptions there (see the ).

But as soon as you think a bit further like what we had "the default value for Attribute B is the value of attribute A if B is not set" or any other more complex "Initialization" logic of a cluster. But thats (beside this one example) more theoretical.

So we can start with option 1, but I personally assume we will need option 2 later too ... (but just stomach feeling ;-) )

[Apollon77]

PS: oohh thought about it ... beside just attribute change handers we also need options to store data on "that instance" level, like in the OnOff case Timers (yes normal Timeouts right now, will change to Timers before we merge)

[Apollon77]

Addition: Groups Cluster needs to have an "own list of groups for the "device" relative to the asking devcie" ... so we needan "instance variable" for that list ... and that also should be possible to be persisted when the server is closed and restored on next start

[Apollon77]

Additionally, as already started to be discussed we need to think about how to access other clusters of the same endpoint.

• OnOffCluster would in my eyes need access to LevelCluster, if existent, to allow "offWithEffect" command (dimming/fading) - if no LevelCluster exists I would say that e.g. this command is not available
• OnOffCluster needs to trigger SceneController to store and recall the state of all clusters (so most likely SceneController needs a method to store/recall which is then iteratingover all clusters of the endpoint for their data)
• LevelCluster also set on/off state ( e.g. brightness=0 -> off) and such interactions. So they both interact with each other
• GroupsCLuster needs access to Identify (query if an identify logic is currently active - access to the time attribute) and Scene Cluster (delete some scenes based on login with group id, so internal access kind of)

Just two things I stumbled over already.

The last idea to also give a refernce to such clusters ia a "config" of the Cluster would end in circular references and partially an issue what to initialize first.

I know you wanted to separate things but my feeling goes into the direction that a cluster should know his "parent endpoint" and allows to access other cluster instances via (pseudcode) "this.parentEndpoint.getClusterInstancesOfType(SceneCluster) == [...]" or auch

[mfucci]

I think we need to introduce a layer to represent devices aka endpoints.

They are also standardize and the behavior of each cluster depends on which device they are part of.
For instance an ON_OFF_PLUGIN_UNIT device contains the OnOffCluster but does not implement the dimmable attributes / commands.
Or a DIMMABLE_LIGHT device makes mandatory the dimmable attributes / commands.

An user of the library will probably manipulate higher level "device" rather than low level clusters.

In this case, all clusters are implemented inside a same device and there is no problem to have cross-references or a common state.

[Apollon77]

They are also standardize and the behavior of each cluster depends on which device they are part of.

Not really ... In fact clusters just know their attributes and featuremaps and such. So yes These depend on the device type, but the cluster itself not from what I saw so far.
Maybe this way we still stay "separated code wise"

But I agree that an Endpoint layer is needed together with "a cluster can get access to cluster instances on the same endpoint". We also might have an "internal API" on the cluster instances (or we call commnds internally as it would be called from external or set attributes - but I think we need both)

Or a DIMMABLE_LIGHT device makes mandatory the dimmable attributes / commands.
Exactly ... this is all defined in the specs - which device type has which clusters mandatory and optional and such. So we could also "define" this in the deviceType ...

An user of the library will probably manipulate higher level "device" rather than low level clusters.
For this we would need to add a higher level devcie specific API ... I would potentially move this to later and for now offer a "device.getClusters(OnOffCLuster).attributes.onoff.get()" ... at least until we have more practical requirements.

For me as "planning to use it" I will contruct for server usage all the clusters anyway and puzzle together the devices, so I have cluster access anyway and will map it. For client usecase I also need to "Parse" the device structure by checking all clusters that are relevant and doing stuff.

[Apollon77]

The more I dig into it the more it feels to me that the "Cluster Server Implementation" needs to be a real class. Yes we can define that it has methods that are named like the commands (or having them with a "command* prefix" as name or such), but these objects also need to provide normal methods and kind of define an "Internal usable API" for inter-cluster-logic.

[Apollon77]

Next thing (sorry that it is that much currently, but I hope we have all cases soon so that I can implement stuff on my own) :-)

I started looking into the "Identify" Cluster. Here the question comes up how we want to have it. In fact when "identify command" is triggered a "timer" is started for X seconds to execute the "identify method". When timer is 0 then the identify method should stop.
The question is in my eyes how "convenient" we ant to do this.

• Option A: We do no implement anything on our side and the "dev that uses the lib" needs to implement all logic
• Option B: We implement the timer logic and have "additional" listeners for the dev to implement to "start/stop identification logic" - but we conveniently handle it for him.
• Option C: We use the examples in the specs (especially for effects) and check if we have a Level/Color Cluster and also "do the effects" and just allow the user to override it with own implementation - but thats very specific and we could only do it for "Identifytype === Light output" ... all others like beep, or small LED or display we are out anyway and needs to be done else.

I think option B makes sense, but requires additional "internal command listeners" that are not 1:1 to the cluster definitoon but "what makes sense". But way more convenient for the developer because all timing handling we do for him.

What you think?

[Apollon77]

The GroupsCluster logic separates between "command was received as a unicast" and "command was received as a groupcast", so we need to transport this info (part of "request" params for the handlers or with an own "communication" struct?) to the handler calls.

[Apollon77]

Ok, I checked the relevant Devicetypes (Root, Bridged Node and Aggregator) and here is the result (optionals in brackets):

Base Cluster Requirements for Matter

• Descriptor Server M
• Binding Server Simple & Client ??
• FixedLabel Server [App & Server & Multiple]
• UserLabel Server [App & Server & Multiple]

Root Node Cluster Requirements

• 0x0028 Basic Information Server I M
• 0x001F Access Control Server I M
• (0x002E Power Source Configuration Server I O)
• (0x0038 Time Synchronization Server I P, O)
• 0x003F Group Key Management Server I M
• 0x0030 General Commissioning Server I M
• 0x0031 Network Commissioning Server !CustomNetworkConfig
• 0x003C Administrator Commissioning Server I M -> TODOs open
• 0x003E Node Operational Credentials Server I M
• (0x002B Localization Configuration Server I LanguageLocale)
• (0x002C Time Format Localization Server I TimeLocale)
• (0x002D Unit Localization Server I UnitLocale)
• 0x0033 General Diagnostics Server I M
• (0x0032 Diagnostic Logs Server I O)
• (0x0034 Software Diagnostics Server I O)
• (0x0037 Ethernet Network Diagnostics Server I [Ethernet])
• (0x0036 Wi-Fi Network Diagnostics Server I [Wi-Fi])
• (0x0035 Thread Network Diagnostics Server I [Thread])

Aggregator Node Cluster Requirements

• (Actions Server O)

Bridged Node Cluster Requirements

• Bridged Device Basic Information Server I M
• (Power Source Configuration Server I BridgedPowerSourceInfo)
• (Power Source Server BridgedPowerSourceInfo)

@mfucci I will now work to provide PRs for all relevant needed Cluster Specs from this list as discussed in our talk

[orlenkoo]

I have tried to test with https://github.com/project-chip/matter.js.
After build, tested with forked with device.js, and I think it works well.

[Apollon77]

Bridged Devices progress

With my local development version I was able to pair a "two sockets bridge" with Google Home. The version is based on #192 and a followup change to InteractionServer.

When doing this you will get a "Bridge"/"Aggregator" device and then the two devices

For other systems maybe more is needed ...

• Alexa reports an error on pairing, no idea if they even support bridged devices right now
• iOS also errors, but here we have other open things where @mfucci is working on

[JimBuzbee]

Nice work! Look forward to seeing it work.

At the moment, using the main branch, I am unable to commission the node-matter device in either Google Home or in Alexa. I can only commission in SmartThings which is only minimally functional due to always sending the "on" command.

[Apollon77]

PS: I also enhanced the executable to allow specifying the type (socket vs light). ANd a new matter-bridge executable would come for bridged where you can define all that also for multiple devices (dynamic number - just the commandline wil get loooooooong :-))

[JimBuzbee]

With regard to the main branch, I've probably messed something up in my environment :-) Will keep playing with it.

Long command line? Well evidently, bash allows 4096 characters - I could hit it as I have a lot of devices on my net! :-)

[Apollon77]

if you hit it we will find other options like a json config file :-) but one step after the other

[orlenkoo]

I have resolved that issue with some tricking with branches.
There are 2 branches - main, add-subrepo-for-node-matter
First, on add-subrepo-for-node-matter branch, did npm install.
After that, checkout to main branch, and did npm install..
And npm run build-clean worked perfectly.

I think all contents on two branches are different completely, especially such as package.json.
So it crashed during npm install on main branch..
Maybe, we can resolve that issue copy package.json from add-subrepo-for-node-matter branch to main branch.

And node version is also important, I am using node16.10.0.
When I use other version, it crashed..

So here is full commands.

git checkout add-subrepo-for-node-matter
npm install
git checkout main
npm install
npm run build-clean

It generated dist correctly.

[Apollon77]

@orlenkoo Please just use main branch!! add-subrepo-for-node-matter is some dev stuff that never went into main and also never will, which issues you had with main branch? What errors exactly are there? The issue from Jim is also nearly 3 month ago ... In fact "checkout main and running npm install" already builds the whole project and after this the dist should be there. If not please open an issue and post your "npm install log". thank you

[javierbarellano4v]

hopefully the right discussion group...

Is there support for 'subscription' and 'subscriber interaction' in node matter

Thanks.

[Apollon77]

Ok whats your usecase? We mainly focussed on "device" and there the controller sends the subscribe, so device "just" need to follow it.

For controller usecase I will provide some fixes soon because it is not working for me in the current version from main branch, then I can add an example too ...

[Apollon77]

In general the API that we have right now is very low level and not yet really making sense to use it big scale because this will in any case get bigger changes!

[javierbarellano4v]

@Apollon77 thanks for this information.

Once working , will a controller subscribe to a device via command line option like in node-matter server? Or some other means?

Looking forward to example...

[Apollon77]

honestly ... no idea right now. I personally see these "executables" that we have right now as "reference implementations to pragmatically proof the functionality". Based on this you can build your own software with it. but yes in general we could also allow a controller to "subscribe" to onoff as example with comparable "command execution triggers" as we use in "device" right now.
Once we have persistence implemented the controlling could also be "just the execution of the script once" and yes for subscriptions it stays running ... then it could become comparable like chip-tool. But at least my goal is not to reimplement chip-tool :-)

In general this whole "command line interface" and "exec triggers" are very limited in what we can do with them ... maybe a "brighness level" is possible, but then it already starts to get complicated ... so lets see ....

[javierbarellano4v]

@Apollon77 Thanks, I understand now. Right, let's see.... appreciate your feedback.

[JimBuzbee]

Got a new weather station at home and thought it would be fun to display the current temperature with a node-matter "device" so I implemented the TemperatureMeasurement cluster. It's a very simple cluster - maybe too simple? Google Home and Alexa both start to pair but then both give a generic error at the end. Maybe they don't have support for a stand-alone Temperature device? SmartThings "succeeds" in pairing, but acts much like the on/off device - adds it, but crashes near the end and not functional - i.e. The temperature never shows (screenshot attached). Of course it's likely I haven't implemented the cluster correctly

Thoughts?

https://github.com/JimBuzbee/node-matter/tree/TemperatureMeasurementCluster

[Apollon77]

Cool, Alexa in any case only supports On/Off and Light. Nothing more. But Google should support it. So we would need a log again ... and with having the code I can also check myself.

Would you like to bring in the clusters as PRs? That would be cool ... ideally one PR per Cluster file.

BTW: I found a bug in subscription handling, so also this could be the reason. hopefully have a PR for fix soon too when error handling PR is merged

[JimBuzbee]

OK - Will create three new PRs, one each for Temperature, Illuminance and Humidity clusters, and then I'll see if I can figure out how to do a pull request

git, Typescript, Matter, all new to me :-)

[Apollon77]

Easiest way is: go to node-matter ... use the web ide to "add the needed file", copy the content, scroll down and commit it - Github automatically makes a branch. Then on next screen click "open pull request". describe what you did ... done

BTW: An "empty" (aka no logic in it) server/* File is not needed

[JimBuzbee]

Pull request in - Also, after put it in, got Thermometer to work on Google Home!

Had forgotten that productID was part of the info that had to be put in the Google Developer's console and I had changed it to 8001 instead of 8000. Once I changed it back, it succeeded! Will play around a bit to see if it is reading properly.

For future reference - Running "adb logcat" and looking for matter messages showed me the problem