Use WinFS-style graph model instead of block tree model

[0xDing]

Problem

Articulate the problem that this piece of work addresses. What is the status quo and why does that not work? Why does the problem matter?

We currently use a Tree structure to store the parent-child relationships between blocks and add additional tables to store structure data such as formulas. This structure is not enough for no-code and automation workflow scenarios. There may be several types of relationships between different blocks or between blocks and other structural data. For example, what content a PageBlock contains, what the parent Page of a PageBlock is, and to which PageBlocks a PageBlock is backlinked are all different types of relationships.

We can build a generic model to represent the relationship between different data.

Why is this the right time to address this problem?

As a refactoring of the core, this epic needs to be completed early to reduce compatibility issues.

Time Budget

2 weeks

Solution

Resource will be used as a generic term for blocks, formulas and other structured data.

• Use json-schema to define ResourceSchema.
  • The PageSchema must have a "render" attribute to specify whether it should be rendered as a document or a canvas or something else.
• Use the Resource model instead of the DocsBlock and DocsFormula models.
• Create a ResourceLink model with subject, predicate and object fields.
  • predicate is an enum value that can be contents, link, parentPage, etc.

Out of Bounds

For the time being we will only refactor the database schema and model, the GraphQL API will remain unchanged. And the tables active_storage_attachments , docs_share_links, etc remain unchanged for the time being.

Rabbitholes

• LinkBlock will be affected by this refactoring.
• Need to consider the impact on CRDT.

Ref

• https://winfs-sdk-docs-example.corp.brickapis.com/html/1c1ff342-9b5e-4641-8f10-5a210acb2b12.htm
• https://www.w3.org/RDF/
• https://blockprotocol.org/

[stackia]

owner @aligo

[aligo]

Combining the WinFS-style model with our next phase Project Q, to achieve object(resource), links, and to carry our existing functionality, the server-side data will be presented by following models:

Resources(Objects):

field	description	type
id	object id	uuid
type	Document/Spreadsheet/Formula	Enum
data	key-value data of object (defined by json-schema of each types)	JSONB
keywords	for full text indexing, extract from data based on object type	text
state_id	CRDT state id	uuid
pod_id	pod id	id
-	deleted_at, created_at, updated_at, and etc	timestamp

Notes:

• All data of the object (for Document, title and content), will be stored in json field data
• To implement CRDT, data of the object, all data of links which subject is this object, will be persisted in states table

---

Links:

field	description	type
subject_id	object id in objects table	uuid
object_id	object id in objects table	uuid
predicate	link type between subject and object	Enum
link_id	(put in meta) optional link id to query by condition	uuid
meta	key-value data indexed to query by condition	JSONB
data	key-value data of link (defined by json-schema of each types)	JSONB
state_id	CRDT state id	uuid
keywords	for full text indexing, extract from data based on object type	text
pod_id	pod id subject	id
-	deleted_at, created_at, updated_at, and etc	timestamp

Notes:

• For example, a user commenting on a Document should also be considered a link, where subject is the document and object is the user, and the content of comment is in data
• For Spreadsheet, Document Object -link-> Spreadsheet Object -(Spreadsheet links)-> Formula (Cell) Object, meta of Spreadsheet links have row number(sort)

---

Because under the current architecture, data used/operated in the client is actually built based on state (rather than data of server-side objects and links), so to ensure full CRDT compatibility (including undo/redo), this planning implement is to include links as a part that can be CRDT manipulated, so all the links of an object whose subject is the object will be presented in the state as object's sub-document

States(of Object)

field	description	type
id	state id	uuid
object_id	object id	uuid
state_type	Full/Update	Enum
state	CRDT state of object data and links	Binary
history_id	history id of this data	uuid
pod_id	pod id	id
-	created_at and etc	timestamp