New feature: individually hydrate/dehydrate extensions

[MtDalPizzol]

Hi, folks! I've just implemented a feature that makes a lot of sense to me and I'd like to know if anyone else has interest in this so we can maybe merge this to the main project.

I couldn't find another way to do what I needed, so I took the route described below, but if there's already a better way to do this, please, point it out to me.

What I needed

1. I have a legacy system that keeps images on a table.
2. These images are then referenced inside the contents of an article.
3. I don't want to save the full URL for the images on the article contents, only the id for the images. This is because if I need to change the domain that the images are served from, I don't need to change this in the database.
4. So, before saving to the database, I need to dehydrate the extension to only save the type of the block (image), and the image's id.
5. When it's time to fill the form, I fetch each image from the table and hydrate the extension to be handle in the frontend.

What I did

I've added the possibility to hydrate/dehydrate extensions individually.

Why I did this?

We have field hydration/dehydration in Filament itself, but, in the case of the editor, this happens only for the entire content of the field.

At the moment, if we need to mutate the data for a specific extension before saving it to the database and when we fill the form on the frontend, the only whey to do this is via the afterStateHydrated, formatStateUsing and the dehydrateStateUsing methods, which needs to be called on every instance on the editor that needs this.

Also, when dealing with this methods, we need to loop over the contents and conditionally handle each block depending on its type, which makes the code clunky and a bit more difficult to be organized. Also, when dealing with this, every developer will end up with something very similar that could be provided out of the box, saving a lot of time and effort for everyone.

What I wanted

I wanted a way to isolate the logic for hydrating/dehydrating each extension, keeping the code organized.

What I came up with?

I created the concept of a Hydrant, not sure if we should call it Hydrator, but you get the idea... This is a class that has 4 methods that are explained below:

<?php

namespace FilamentTiptapEditor\Extensions\Hydrants;

use Illuminate\Support\Facades\Config;

class ImageHydrant
{
    protected $ids = [];
    protected $records;

    /**
     * Register is called once for each block
     * to keep track of the ids in the hydrant itself.
     * The ids are later used in the preHydrate method
     * to fetch all the records we need from the database at once.
     */
    public function register($block) {
        $this->ids[] = $block['id'];
        
        $this->ids = array_filter($this->ids);
    }

    /**
     * preHydrate runs once before we loop through the blocks again.
     * Here we can fetch all the records we need in one go,
     * instead of fetching them one by one in the hydrate method.
     * This gives us a huge performance boost,
     * since we're hitting the database only once.
     */
    public function preHydrate() {
        $this->records = Config::get('curator.records_media_model')::whereIn('id', $this->ids)->get();        
    }

    /**
     * hydrate runs once for each block.
     * Here we can fetch the record we need from the records we fetched in preHydrate.
     * We can also run any other logic we need for each block.
     */
    public function hydrate($block) {
        if (!isset($block['id'])) {
            return;
        }

        $item = $this->records->where('id', $block['id'])->first();

        return [
            'attrs' => [
                'id' => $item->id,
                'src' => $item->media->url,
                'alt' => $item->media->alt,
                'title' => $item->media->title,
                'width' => $item->media->width,
                'height' => $item->media->height,
                'lazy' => false,
            ]
        ];
    }

    /**
     * dehydrate runs once for each block.
     * Here we can run any logic we need for each block
     * before it's saved to the database.
     */
    public function dehydrate($block) {
        return [
            'id' => $block['attrs']['id']
        ];
    }
}

Then, we can register any custom Hydrant in the config file, where the key in the array is the type of the block:

    // config/filament-tiptap-editor.php
    // ...
    'extension_hydrants' => [
        'image' => FilamentTiptapEditor\Extensions\Hydrants\ImageHydrant::class,
    ]

From here, we change the setUp method on TiptapEditor.php to call the methods on the Hydrant at the points need. For instance, before calling the renderPreviewsMethod, we call handleExtensionsHydration.

    protected function handleExtensionsHydration(array $document): array {

        $this->registerExtensionsForHydration($document);

        $this->preHydrateExtensions();

        return $this->hydrateExtensions($document);
    }

    protected function registerExtensionsForHydration(array $document): void {
        $content = $document['content'];

        foreach ($content as $k => $block) {
            if (array_key_exists('content', $block)) {
                $this->registerExtensionsForHydration($block);
            } else if ($hydrant = $this->getHydrant($block['type'])) {
                if (method_exists($hydrant, 'register')) {
                    $hydrant->register($block);
                }
            }
        }
    }

    protected function getHydrant($type) {
        if (isset($this->hydrants[$type])) {
            return $this->hydrants[$type];
        }

        $hydrant = config('filament-tiptap-editor.extension_hydrants.' . $type);

        if (!$hydrant) {
            return;
        }

        $hydrant = app($hydrant);

        $this->hydrants[$type] = $hydrant;
        
        return $this->hydrants[$type];
    }

    protected function preHydrateExtensions() {
        foreach ($this->hydrants as $key => $hydrant) {
            $hydrant->preHydrate();
        }
    }

    protected function hydrateExtensions(array $document): array {
        $content = $document['content'];

        foreach ($content as $k => $block) {
            if (array_key_exists('content', $block)) {
                $content[$k] = $this->hydrateExtensions($block);
            } else if ($hydrant = $this->getHydrant($block['type'])) {
                if (method_exists($hydrant, 'hydrate')) {
                    $content[$k] = array_replace_recursive(
                        $block,
                        $hydrant->hydrate($block) ?? []
                    );
                }
            }
        }
        
        $document['content'] = $content;

        return $document;          
    }

Let me know what you think about this, and specially if there's a better way to achieve this that I don't know about.

[awcodes]

Is this for a custom image block?

[MtDalPizzol]

It depends... That's something that was unclear to me in the README. There's Custom Extensions AND Custom Blocks, and the difference wasn't a lot clear for me. But, yes... the goal is to have image blocks on the content, but the way I store them in the database is a lot different than the one that the default Image Extension does.

[awcodes]

Ok, custom extensions are for Tiptap itself. Blocks are a way to insert custom rendered blade components into the editor's content.

[awcodes]

I think the best solution to this would be an option in Curator to allow for relative paths on the insertMedia action. Then you wouldn't have to go through all of this.

I am impressed by it all though. 😃

[awcodes]

The editor plugin already has a similar option for its native media action.

[MtDalPizzol]

I think the best solution to this would be an option in Curator to allow for relative paths on the insertMedia action. Then you wouldn't have to go through all of this.

I am impressed by it all though. 😃

Yeah, but then we would be talking only about the Curator case. I went through all this "trouble" because I already have a bunch of other custom extensions unrelated to Curator that will need that kind of handling.

Also, this is totally optional... You add Hydrants only if you need them. And I think it keeps the code very clean, organized and it's very simples to customize any new extensions I add.

But anyway... let me know if you'd like to take this further. Otherwise, I'm happy enough for achieving this and keeping this in my isolated branch. 😃

[awcodes]

I could just not be fulling understanding too.

Is this for blocks only?

[MtDalPizzol]

I'm using this for custom extensions, specifically. I'm not sure if this will be useful for custom blocks or other stuff.

Let me try too be more clear about the reason for this, using Curator as an example.

When you insert an image selected via Curator, then save the resource that contains the TiptapEditor field, the column where the field is saved will store something like this:

{
   "type":"doc",
   "content":[
      {
         "type":"paragraph",
         "attrs":{
            "textAlign":"start",
            "class":null
         },
         "content":[
            {
               "type":"image",
               "attrs":{
                  "url":"http://assets.mydomain.com",
                  "width":"100",
                  "height":"100",
                  "title":"Image title",
                  "alt":"Image alt"
               }
            }
         ]
      }
   ]
}

Now... I'm dealing with a legacy system that stores the information about the image in a dedicated table, and I need to keep that data centralized on that table. When an image is updated on the table, the image is rendered with the updated data on every content that references that image.

So, instead of having that data hardcoded on the content saved by the editor, I need the editor to only store a reference to the image on the legacy table. Something like this:

{
   "type":"doc",
   "content":[
      {
         "type":"paragraph",
         "attrs":{
            "textAlign":"start",
            "class":null
         },
         "content":[
            {
               "type":"image",
               "id":2
            }
         ]
      }
   ]
}

Another custom extension I have that uses the same approach is for ad units. I have a bunch of ad creatives that are attached to an ad unit. When I save the resource, the editor only needs to save the following:

{
   "type":"ad_unit",
   "id":"123"
}

It doesn't need to store anything about the ad creatives, like image or video URLs, because the creatives attached to an ad unit will change over time. All the editor needs to know is that there's an ad unit at some point in the middle of the contents and let me handle what will be previewed/rendered inside that block at any moment in time, probably using a Tip Tap Interactive Node View.

So, I needed a way to strip all that information that the editor needs to render the preview for the image before saving the contents to the database (dehydration, on Filament's jargon). And then, I needed a way to reverse that process when it's time to fill the form again (hydration, on Filament's jargon again).

Now, the problem is that dehydration/hydration happens for the editor field as a whole, not individually for extensions. So, in order to handle objects with "type": "image", I would need something like this:

  TiptapEditor::make('structure')
        ->output(TiptapOutput::Json)
        ->afterStateHydrated(function ($state) {
            $content = $state['content'];

            foreach ($content as $k => $block) {
                if ($block['type'] === 'image') {
                    // fetch the image from the DB
                    // format the array to be returned
                }

                // If I have other custom extensions that need handling
                // I need more conditionals and the code start to get cluthered
                if ($block['type'] === 'image') {
                    // fetch data from the DB
                    // format the array to be returned
                }
            }

            $document['content'] = $content;

            return $document;
       })
       ->dehydrateStateUsing(function ($state) {
            $content = $state['content'];

            foreach ($content as $k => $block) {
                if ($block['type'] === 'image') {
                    // strip the unwanted information
                    // format the array to be saved
                }

                // If I have other custom extensions that need handling
                // I need more conditionals and the code start to get cluthered
                if ($block['type'] === 'image') {
                    // strip the unwanted information
                    // format the array to be saved
                }
            }

            $document['content'] = $content;

            return $document;
       })

Now, since afterStateHydrated and dehydrateStateUsing are called internally by TiptapEditor.php, I've modified those calls to handle the logic I presented when I started this discussion.

That way, I can have all the logic needed for hydrating/dehydrating each extension contained in its own file, and don't need to worry about handling the editor field as whole, but concentrate on what each extension puts into the database and gets back from it without having to modify anything inside the package.

For instance:

<?php

namespace FilamentTiptapEditor\Extensions\Hydrants;

use Illuminate\Support\Facades\Config;

class ImageHydrant
{
    protected $ids = [];
    protected $records;

    public function register($block) {
        $this->ids[] = $block['id'];
        
        $this->ids = array_filter($this->ids);
    }

    public function preHydrate() {
        $this->records = Config::get('curator.records_media_model')::whereIn('id', $this->ids)->get();        
    }

    public function hydrate($block) {
        if (!isset($block['id'])) {
            return;
        }

        $item = $this->records->where('id', $block['id'])->first();

        return [
            'attrs' => [
                'id' => $item->id,
                'src' => $item->media->url,
                'alt' => $item->media->alt,
                'title' => $item->media->title,
                'width' => $item->media->width,
                'height' => $item->media->height,
                'lazy' => false,
            ]
        ];
    }

    public function dehydrate($block) {
        return [
            'id' => $block['attrs']['id']
        ];
    }
}

<?php

namespace FilamentTiptapEditor\Extensions\Hydrants;

use App\Models\AdUnit
use Illuminate\Support\Facades\Config;

class AdUnitHydrant
{
    protected $ids = [];
    protected $records;

    public function register($block) {
        $this->ids[] = $block['id'];
        
        $this->ids = array_filter($this->ids);
    }

    public function preHydrate() {
        $this->records = AdUnit::whereIn('id', $this->ids)->get();
    }

    public function hydrate($block) {
        if (!isset($block['id'])) {
            return;
        }

        $item = $this->records->where('id', $block['id'])->first();

        return [
            'attrs' => [
                'id' => $item->id,
                'width' => $item->media->width,
                'height' => $item->media->height,
            ],
            'creatives' => $item->creatives->toArray()
        ];
    }

    public function dehydrate($block) {
        return [
            'id' => $block['attrs']['id']
        ];
    }
}

Hope it makes it clearer....

[awcodes]

I wonder if this would be better on the TiptapBlock class, that way each block can handle it's own hydration / dehydration instead of the field having to do it.

[awcodes]

Yea, tiptap-php is for parsing the content, and there's a TiptapBlock extension the parses it. The plugin uses tiptap-php to parse the content in and out of the db by converting it to either JSON or HTML.

I'm just thinking that the block itself can handle the hydration / dehydration of it's data so that when the field loops the content to handle blocks the block could call it's own $block->hydrate() / $block->dehydrate() methods, before modifying the ast tree that gets returned to the callbacks in the setUp.

Just trying to make sure the tree doesn't have to get looped twice.

[MtDalPizzol]

I like this idea... Every time we can avoid introducing new concepts its a good thing and I also think that keeping this logic on the block will be better. I'm just thinking that if we avoid looping twice, we will need to hit the database multiple times since we're not getting a chance to collect all the ids beforehand.

[awcodes]

That's a valid point. Something to think about. I am interested in this though. Definitely an edge case in my opinion but I can see value in it.

[MtDalPizzol]

I'll dig deeper into it... try to achieve this via the TiptapBlock class. Maybe we can have a public static preHydrate method on the class, but I don't see how to avoid looping twice. Another thing is this: in the case for the Image extension, it doesn't inherit from the TiptapBlock class, it inherits from Tiptap\Nodes\Image. So, at moment I'm not sure if we'll benefit from restricting this to the concept of blocks.

[awcodes]

True. Would hate to have to maintain copies of every extension, node and mark in the Tiptap-php package to extend them all.

In reality this is a content manager, and not a relationship manager. 🤔