Provide a way for custom Resources to be versioned

[blackears]

I would like to propose adding a way to add versioning information for custom resources and also a way to load older version into a newer version of the resource.

As a project becomes more mature, the data you produce with it becomes more important to the project. While in the early stages you might be able to just throw out all your old data if you radically change the design of your resources, if you have a mature project in which you've designed levels and assets that rely on your custom resources and resource editors, you want to be able to preserve that data as much as you can.

This is currently not really possible in Godot. While some minor changes to a custom resource - such as turning int into a float - may not cause an issue, if you introduce new child objects and remove old ones or even just rename things, you've pretty much just lost all the data you're storing in them. This is particularly true when you're authoring addons and can't really tell everyone using your addon that they have to throw out everything they've made with it because you've just revised one of the core data structures.

I'd like to suggest adding a @version tag to GDScript that can be added to the start of any resource file to declare it as a specific version.

@version("2")
extends Resource
class_name Player

@export var weapon:int #weapon id
@export var position:Vector2i #location in the world
@export var image:Texture2D  #picture

Then if you alter the code in a later revision, you can change the version identifier

@version("3")
extends Resource
class_name Player

@export var weapon:WeaponResource #weapon
@export var position:Vector2i #location in the world
@export var image:Texture2D  #picture

Resources without this tag will be assigned the version of the empty string.

To allow for dealing with old versions, I'd suggest also providing a VersionedResourceLoader class that the editor would use to load out of date versions of a resource:

extends VersionedResourceLoader
class_name PlayerLoader

var image_symbol_id:int

#Returns true of the loader can handle this 
func _can_load(class_string_name:String, version:String)->bool:
    return class_string_name == "Player" && version == "2"
    
func _create_resource(params:Dictionary):
    var resource = Player.new()
    
    #Set up resource based on parameters in dictionary
    resource.position = params["position"]
    resource.weapon = WeaponResource.new()
    resource.weapon.id = params["weapon"]
    
    image_symbol_id = params["image"]
    
    return resource

func _initialize_resource(resource, resource_symbol_table:Dictionary):
    #finish setting up resource based on resource symbols
    resource.image = resource_symbol_table[image_symbol_id]

The _can_load function checks if the loader can handle the given class and version number. The create_resource method has a dictionary listing parameter name/value pairs, with the exception of child resources where the resources are replaced with symbol table ids. (This happens for all the child resources too.) Then the resource finishes its creation in the _initialize_resource function where the table mapping all the symbol ids to the created resources allows for the set up to complete.

[Calinou]

Can't you add _get() and _set() methods to create compatibility handlers? This is what we do internally when renaming properties.

For example, in C++, this is: https://github.com/godotengine/godot/blob/6da4ad16624484398331f393b503f8b5e2888c51/scene/resources/box_shape_3d.cpp#L60-L76

_set() is only required if you wish users to be able to set the old property name on it. For one-way compatibility, you only need to implement _get().

[blackears]

I suppose I hadn't considered that. But what if you completely change the data format? For example, I have a class that currently has some explicitly declared arrays

extends Resource
class_name ConvexBlockData

@export var vertex_points:PackedVector3Array  #Per vertex
@export var vertex_selected:PackedByteArray  #Per vertex

@export var edge_vertex_indices:PackedInt32Array
@export var edge_selected:PackedByteArray
@export var edge_face_indices:PackedInt32Array

@export var face_vertex_count:PackedInt32Array #Number of verts in each face
@export var face_vertex_indices:PackedInt32Array  #Vertex index per face
@export var face_material_indices:PackedInt32Array #Material index for each face
@export var face_uv_transform:Array[Transform2D]
...

I'd like to replace this with a system where I can add or remove data planes as I need them. So I'd like to delete all these arrays and have something like

extends Resource
class_name ConvexBlockData

@export var vertex_data:Array[DataPlane]
@export var edge_data:Array[DataPlane]
@export var face_data:Array[DataPlane]

-----------------

extends Resource
class_name DataPlane

@export var name:String
@export var type:Variant.Type
@export var data:Array

So this isn't replacing old parameters with new ones but a completely new way to structure things. Can I get this to work with the _get() method?

[Calinou]

So this isn't replacing old parameters with new ones but a completely new way to structure things. Can I get this to work with the _get() method?

Yes, but you need to rename the property in the process.