Improve Image Uploading Experience

[steven-tey]

Today, we're launching Image Uploads for Novel:

CleanShot.2023-06-21.at.18.04.54.mp4

Images are uploaded and stored in Vercel Blob.

However, this is not the ideal UX...yet.

Potential Improvement

Would love to know if it's possible to show the local base64 version of the image as a preview while the upload is in progress, and when it's done, replace the base64 url with the uploaded URL.

This is similar to Notion's uploading experience:

CleanShot.2023-06-21.at.18.10.08.mp4

I scoured the web and can't seem to find a way to reliably change the src of an image node after the image is fully uploaded. The closes I could find is this comment but I couldn't figure out how to make it work 😅

Would love to know if other folks have any ideas on how to implement this!

[hernancito]

Hi Steven! I'm sharing a demo and the code used in our editor to preview the image before switching to the correct URL. It flashes in the swap as Guillermo tweeted https://twitter.com/rauchg/status/1671896959152701440?s=20 😅 but I think I know how to improve it by looking at the Novel repo
Hope this helps, I can do a PR later to give it a try.

image-upload.mov

A quick explanation

I did it by following this code https://gist.github.com/slava-vishnyakov/16076dff1a77ddaca93c4bccd4ec4521?permalink_comment_id=3768989 (below is the updated version to work with Tiptap 2.0 https://gist.github.com/slava-vishnyakov/16076dff1a77ddaca93c4bccd4ec4521#gistcomment-3744392)

1. We need a plugin to handle the Prosemirror events (cursor position, URL swapping)

UploadPastedImagesPlugin.tsx

import { Plugin, PluginKey } from '@tiptap/pm/state';
import { Decoration, DecorationSet } from '@tiptap/pm/view';

const uploadKey = new PluginKey('history');
export type UploadFn = (image: File) => Promise<any>;

const UploadPastedImagesPlugin = (upload: UploadFn) =>
  new Plugin({
    key: uploadKey,
    state: {
      init() {
        return DecorationSet.empty;
      },
      apply(tr, set) {
        // Adjust decoration positions to changes made by the transaction
        set = set.map(tr.mapping, tr.doc);
        // See if the transaction adds or removes any placeholders
        const action = tr.getMeta(this);
        if (action && action.add) {
          // Lower image opacity to indicate uploading process
          const widget = document.createElement('img');
          widget.setAttribute('class', 'opacity-40');
          widget.src = action.add.src;
          const deco = Decoration.widget(action.add.pos, widget, {
            id: action.add.id,
          });
          set = set.add(tr.doc, [deco]);
        } else if (action && action.remove) {
          set = set.remove(
            set.find(null, null, (spec) => spec.id == action.remove.id)
          );
        }
        return set;
      },
    },
    props: {
      decorations(state) {
        return this.getState(state);
      },
      handleDOMEvents: {
        paste(view, event) {
          pasteHandler(view, event, upload);
        },
      },
    },
  });

const pasteHandler = (view, event, upload) => {
  // Get the data of clipboard
  const clipboardItems = event?.clipboardData?.items;
  if (!clipboardItems) return false;
  const items = Array.from(clipboardItems).filter((item: any) => {
    // Filter the image items only
    return item.type.indexOf('image') !== -1;
  });
  if (items.length === 0) {
    return false;
  }

  const item: any = items[0];
  const file = item.getAsFile();
  if (!file) {
    return false;
  }
  if (event?.clipboardData?.types.includes('text/rtf')) {
    // Do not convert pasted rtf to image
    return false;
  }
  startImageUpload(view, file, upload);
  event.preventDefault();
  return true;
};

function findPlaceholder(state, id) {
  const decos = uploadKey.getState(state);
  const found = decos.find(null, null, (spec) => spec.id == id);
  return found.length ? found[0].from : null;
}

function startImageUpload(view, file, upload) {
  // A fresh object to act as the ID for this upload
  const id = {};

  // Replace the selection with a placeholder
  const tr = view.state.tr;
  if (!tr.selection.empty) tr.deleteSelection();

  const reader = new FileReader();
  reader.readAsDataURL(file);
  reader.onload = () => {
    tr.setMeta(uploadKey, {
      add: {
        id,
        pos: tr.selection.from,
        src: reader.result,
      },
    });
    view.dispatch(tr);
  };

  upload(file).then((src) => {
    const pos = findPlaceholder(view.state, id);
    // If the content around the placeholder has been deleted, drop
    // the image
    if (pos == null) return;
    // Otherwise, insert it at the placeholder's position, and remove
    // the placeholder
    view.dispatch(
      view.state.tr
        .replaceWith(pos, pos, view.state.schema.nodes.image.create({ src }))
        .setMeta(uploadKey, { remove: { id } })
    );
  });
}

export default UploadPastedImagesPlugin;

2. We will use this plugin to extend our Image component. It is almost a copy of image.ts from tiptap

import { Node, nodeInputRule } from '@tiptap/core';
import { mergeAttributes } from '@tiptap/react';
import UploadPastedImagesPlugin, { UploadFn } from './UploadPastedImagesPlugin';

interface ImageOptions {
  inline: boolean;
  HTMLAttributes: Record<string, any>;
}

const IMAGE_INPUT_REGEX = /!\[(.+|:?)\]\((\S+)(?:(?:\s+)["'](\S+)["'])?\)/;

const ImageExtension = (uploadFn: UploadFn) => {
  return Node.create<ImageOptions>({
    name: 'image',

    addOptions() {
      return {
        inline: false,
        HTMLAttributes: {},
      };
    },

    inline() {
      return this.options.inline;
    },

    group() {
      return this.options.inline ? 'inline' : 'block';
    },

    draggable: true,

    addAttributes() {
      return {
        src: {
          default: null,
        },
        alt: {
          default: null,
        },
        title: {
          default: null,
        },
      };
    },
    parseHTML: () => [
      {
        tag: 'img[src]',
        getAttrs: (dom) => {
          if (typeof dom === 'string') return {};
          const element = dom as HTMLImageElement;

          const obj = {
            src: element.getAttribute('src'),
            title: element.getAttribute('title'),
            alt: element.getAttribute('alt'),
          };
          return obj;
        },
      },
    ],
    renderHTML: ({ HTMLAttributes }) => [
      'img',
      mergeAttributes(HTMLAttributes),
    ],

    addCommands() {
      return {
        setImage:
          (attrs) =>
          ({ state, dispatch }) => {
            const { selection } = state;
            const position = selection.$head
              ? selection.$head.pos
              : selection.$to.pos;

            const node = this.type.create(attrs);
            const transaction = state.tr.insert(position, node);
            return dispatch?.(transaction);
          },
      };
    },
    addInputRules() {
      return [
        nodeInputRule({
          find: IMAGE_INPUT_REGEX,
          type: this.type,
          getAttributes: (match) => {
            const [, alt, src, title] = match;
            return {
              src,
              alt,
              title,
            };
          },
        }),
      ];
    },
    addProseMirrorPlugins() {
      return [UploadPastedImagesPlugin(uploadFn)];
    },
  });
};

export default ImageExtension;

3. Use the new ImageExtension in the editor 🎉:

import ImageExtension from './ImageExtension';

const savePastedImage = async (file: File) => {
...
}

new Editor({
  extensions: [
    ...
    ImageExtension(savePastedImage),
    ...

[hernancito]

Thank you! I really appreciate your comment ❤️
I started working on the PR, adapting it to Novel

[steven-tey]

@hernancito can't wait! :star-struck_cat:

Does the implementation support multiple images btw? (related issue: #69)

[hernancito]

I think we have a decent implementation. Here are demo (throttling applied on purpose because I don't have a Vercel Blob token).

novel-placeholder.mov

And here I pasted an image between two blocks of text.

novel-placeholder-text.mov

There is a lot of room for improvement, such as avoiding adding a new line when pasting the image into an empty block

[steven-tey]

Incredible work @hernancito! Just merged the PR, and announced it on Twitter / LinkedIn!

Thank YOU for the PR!

[hernancito]

I really appreciate your shoutout @steven-tey! Thank you for your great work too

[pySushi]

Please include options to resize by dragging edges and image-fit options