accessing File's publicURL field resolver without using gatsby-source-filesystem for local content

[pieh]

When using wordpress or drupal source plugins and not using gatsby-source-filesystem for local files publicURL field is not available. To get access to it we need to add gatsby-source-filesystem to gatsby-config.js even if we don't need it to pull local files data. My issue is that we need to specify path for gatsby-source-filesystem to watch. I've used this entry as a kind of workaround

    {
      // We need filesystem source plugin to add publicURL function to File nodes
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `placeholder`,
        // path is required param, so let's just point it to single file to not create
        // much unnecessary work for it
        path: `${__dirname}/gatsby-config.js`,
      },

Possible solutions:

• move publicURL field resolver to separate plugin
• add some kind of flag to gatsby-source-filesystem to skip watching files

I think moving to separate plugin makes more sense because in situation described above we use that plugin not as a source but as extender of File node (so this is mostly based on source/transformer convention). After moving it to separate plugin we could maintain this feature in gatsby-source-filesystem by depending on newly created package to not break sites using it.

Thoughts?

[m-allanson]

With this and #4904, maybe there's a case for useful plugin tools? gatsby/plugin-utils or something.

[KyleAMathews]

Agree about splitting out publicURL to its own package. I'm not sure if I like the idea of a generic "utils" library though as they tend to accumulate an odd assortment of tools of varying quality and usage. Single-purpose packages tend to stay much more focused and lean.

[m-allanson]

I'm not sure if I like the idea of a generic "utils" library though as they tend to accumulate an odd assortment of tools of varying quality and usage.

I nearly wrote something similar. 👍 to a single purpose package.

[KyleAMathews]

Due to the high volume of issues, we're closing out older ones without recent activity. Please open a new issue if you need help!

[jessevdp]

I recently ran into this problem while writing a custom source plugin for a custom API of mine. I have some images that I wanted to import via my source plugin, so I used createRemoteFileNode from gatsby-source-filesystem as the docs suggest. This worked beautifully, every image turned into a File node, but I couldn't access the images publicly (via publicURL) because I didn't have the gatsby-source-filesystem active on my gatsby-config because I wasn't actually using it.

I can confirm that the "hack" (of using the plugin and pointing it at the config file) that @pieh suggests works perfectly fine. But I really dislike the fact that I can't bundle it up into my custom plugin. I would need to instruct others to modify their own gatsby config.

To work around this I've implemented a publicUrl property on the File type myself, from my custom plugin. The code for this isn't too complex.

The following code adds a custom publicUrl property to the File type that just copies the file to the public/static directory (if it hasn't been done already) and returns the url for the new location of the file. This mostly resembles the code from gatsby-source-filesystem.

const path = require('path')
const fs = require('fs-extra')

exports.createResolvers = ({ createResolvers, pathPrefix }) => {
  const publicStaticDir = path.join(
    process.cwd(),
    'public',
    'static'
  )

  createResolvers({
    File: {
      publicUrl: {
        type: 'String',
        description: `Copy file to static directory and return public url to it`,
        resolve: async src => {
          const { absolutePath } = src
          const { ext, name } = path.parse(absolutePath)
          const filename = name + "--" + src.internal.contentDigest + ext
          const newPath = path.join(publicStaticDir, filename)
          
          try {
            const exists = await fs.exists(newPath)
            if (!exists) await fs.copy(absolutePath, newPath)
            return `${pathPrefix}/static/${filename}`
          } catch (e) {
            console.error(`error copying file from ${absolutePath} to ${newPath}`, e)
            return null
          }
        }
      }
    }
  })
}

[jessevdp]

As I'm working on this, I realized there is another workaround. When you're already creating a plugin, or you just have a gatsby-node.js file, you can just reuse the code from gatsby-source-filesystem that adds the publicURL property to the File type.

I got this idea when reading the source code of gatsby-source-filesystem. In their gatsby-node file they export a function setFieldsOnGraphQLNodeType. Which requires a different file extend-file-node.js.

In my own plugin, which already requires gatsby-source-filesystem for createRemoteFileNode, I can just call the following to add the publicURLto theFile` type.

exports.setFieldsOnGraphQLNodeType = require("gatsby-source-filesystem/extend-file-node")

[pieh]

One thing you can do right now (that wasn't possible back when this issue was first created) is to add gatsby-config.js file to your plugin with gatsby-source-filesystem entry (it still need to point to some file).

Gatsby started to support nested gatsby-config files in gatsby@2.13.0 so this would be minimal gatsby version that would support that.

I would not advise to import internals from gatsby-source-filesystem (as those are internals and this can break when when we refactor things)

[jessevdp]

Yeah pointing to internals isn't the greatest idea. But it beats implementing the publicURL myself.

I actually tried putting a gatsby-config in my plugin before. It didn't seem to work. I just tried it again. My gatsby version is 2.16.1.

After removing the import to the internals of gatsby-source-filesystem—I created a gatsby-config.js file in the root of plugins/my-source-plugin/ and put the following config in there:

module.exports = {
  plugins: [
    {
      resolve: "gatsby-source-filesystem",
      options: {
        path: `${__dirname}/gatsby-config.js`,
      },
    }
  ]
}

I first ran gatsby clean and then ran gatsby-develop. It seems like it did not work... I received an error indicating that the publicURL property wasn't working.

Unknown field 'publicURL' on type 'File'.

[jessevdp]

@pieh any idea as to why the nested config isn't working?

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[mjmaurer]

I whipped up a plugin that packages the extend-file-node logic from gatsby-source-filesystem: https://www.gatsbyjs.org/packages/gatsby-transformer-remote-filesystem/

Also has a few options for fingerprinting and filtering by media type

[Vacilando]

I confirm that https://www.gatsbyjs.org/packages/gatsby-transformer-remote-filesystem/ is an excellent temporary solution (until this issue is properly solved by Gatsby) also for files sourced from Drupal.