Inventory Nodes

[c-heat16]

In the previous version of the API, inventory nodes could be queries/indexed like dictionaries - if a material named 'my_material' was added to my_inventory it could be obtained via my_inventory['my_material']. This provided an easy, convenient way to access materials associated with a particular project/collection.

This behavior doesn't seem to have carried over to the new version of the API, however, and the utility that the inventory nodes provides to the user (programmer) is unclear IMO. Admittedly, though, I'm not familiar with the overall objectives of CRIPT, only how I/my team interacts with it. Without the indexing ability, it seems like the inventory is just a collection of materials associated with a project/collection, which could be constructed programmatically without the user - when a project/collection is saved (api.save(project)), the system could just identify material nodes associated with the project and put them in an inventory.

I have found the indexing functionality very useful for my work and have added a make_material_dict() method (below) to the Inventory class which can be called once the inventory is created and populated. For it to work, however, lines 72-75 (__setattr__) of cript/nodes/core.py must be commented out/deleted. While it works, as intended, it goes through a long routine of re-validating each material graph afterwards, which is not necessarily quick with a large inventory. Perhaps that can be improved at some point in the future.

def make_material_dict(self):
    self.material_dict = {}
    for material in self.material:
        print('Updating mapping for {}...'.format(material.name))
        self.material_dict[material.name] = material

[nh916]

hi @c-heat16 this is a great suggestion. I agree being able to query material from an inventory like a dictionary makes it easier. Currently our Inventory node keeps a list of material nodes and the API expects a list of material nodes so I am not sure how to handle that as of right now.

SDK inventory node dataclass

    @dataclass(frozen=True)
    class JsonAttributes(PrimaryBaseNode.JsonAttributes):
        # uuid_base.py inherited attributes
        uuid: str = field(default_factory=lambda: str(uuid.uuid4()))
        updated_by: Any = None
        created_by: Any = None
        created_at: str = ""
        updated_at: str = ""
        
       # primary_base_node.py inherited attributes
        locked: bool = False
        model_version: str = ""
        public: bool = False
        name: str = ""
        notes: str = ""

        # inventory.py attributes
        material: List[Material] = field(default_factory=list)

JSON the API requires

{
   "name":"my inventory name",
   "node":["Inventory"],
   "uid":"_:90f45778-b7c9-4b77-8b83-a6ea9671a937",
   "uuid":"90f45778-b7c9-4b77-8b83-a6ea9671a937",
   "material":[
      {
         "node":["Material"],
         "name":"my material 1",
         "uid":"_:9679ff12-f9b4-41f4-be95-080b78fa71fd",
         "uuid":"9679ff12-f9b4-41f4-be95-080b78fa71fd",
         "bigsmiles":"[H]{[>][<]C(C[>])c1ccccc1[]}"
      },
      {
         "node":["Material"],
         "name":"my material 2",
         "uid":"_:1ee41708-3531-43eb-8049-4bb91ad73df6",
         "uuid":"1ee41708-3531-43eb-8049-4bb91ad73df6",
         "bigsmiles":"654321"
      }
   ]
}

There are a few ways of doing this that I've thought of, but the issue then becomes performance, accuracy, and code maintainability.

What I can do is when the user tries to interact with the inventory to turn the materials list into a dict, o(n) operation, and give it to the user and after they try to set the materials list for the inventory then convert it from dict to list again, another o(n) operation.

Another way, would be that we have a method within the inventory node called get_material_by_name(material_name) and then it does a linear search through the materials list, o(n), to find the material the user specified and give them that, however, setting the node could later become an issue.

The other way would be to keep the list of materials in the inventory as a dict and only serialize and deserialize them into a list, I think this might be the best way to go, but I am not sure right now, and I'll have to figure out serialization and deserialization code for it.

Let me know what your thoughts are as they are very insightful.

for reference here are the current inventory getters and setters

   @property
    @beartype
    def material(self) -> List[Material]:
        return self._json_attrs.material.copy()

    @material.setter
    @beartype
    def material(self, new_material_list: List[Material]):
        new_attrs = replace(self._json_attrs, material=new_material_list)
        self._update_json_attrs_if_valid(new_attrs)

[c-heat16]

Hi @nh916 great, lmk if I can do anything to help.

Just so we're on the same page, the part I mentioned about being 'not necessarily quick' is related to the 'validation' that happens after the material dictionary is created for the inventory. The code I shared above is also O(N), and I guess so is the 'validation' afterwards, but I presumed that validation could be skipped here - the graph has already been validated and we're not changing the underlying structure of the nodes or anything, just putting them in a dictionary. So, if it is safe to skip that here, I think it would be a big time saver. But perhaps it is necessary

[nh916]

Hi @c-heat16, the SDK's performance is a known issue that we are in the process of fixing. Originally when building the SDK we wanted to be sure there are a lot of guard rails for the user, so if there is any error at any point then it is caught immediately so the user is never left wondering why the program is not working and having a hard time debugging it. However, I think we may have done too much validation in that it is slowing the program down. We have a few solutions for this that we are thinking of implementing, which is to just skip validation altogether if needed, to validate only the newly changed nodes and not validate the previously validated nodes, and to try to validate only the important areas and leave the parts that are more secure and optional out of the validation.

The code for the validation that you are hitting exists within api.py _is_node_schema_valid(). The API method is being called whenever the node is converted into JSON within nodes/core.py get_json() either explicitly by the user or under the hood by the SDK. It is also called whenever a node setter is called via the core.py _update_json_attrs_if_valid() method to be sure the new values being set are valid.