Disable storing static uploads locally

[realmacaw]

Is there any way I can disable storing static files locally and use S3 only?

[jmikrut]

Hey @realmacaw — totally!

You can do this completely through hooks. Here is a relevant Discussion that addresses this topic (Cloudinary, but same principles apply).

Also, @richardvanbergen is working on building out an S3 plugin, which would automate all of this for you. We're pumped about that (and not to mention, happy to support how we can).

Plugin progress is briefly mentioned here:

#245

I'm sure more will be coming from that shortly!

[realmacaw]

That's great. Following #249. I was just wondering if the uploads to local storage might cause problems if I deploy the app on AWS ECS on a docker image. I guess having the option to disable local storage when we're opting for cloud storage would be really good right?

[richardvanbergen]

if the uploads to local storage might cause problems if I deploy the app on AWS ECS on a docker image

I'm not sure, digital ocean app platform doesn't seem to state any limit on ephemeral storage. I haven't used EC2 in a long while but I guess the way you would deal with that would be by setting up alerts for your attached storage? I can't remember how it's supposed to work.

I might have one problem with preventing storage on the local FS though. If you use upload.sizes to generate thumbnails then those get generated on the file system and passed through to my plugin. The issue is that before I was grabbing the image from the request and just straight-up uploading them, ignoring the version that is stored on the FS. That's not going to work for the generated thumbnails unless I do my own image processing before uploading.

What do you think @jmikrut and @realmacaw? Should I pull the generated thumbnails from the file system or regenerate them and upload them straight from the PayloadRequest object while preventing storing them locally?

[jmikrut]

@richardvanbergen I think that we could easily append the file sizes to the req, which would then make them usable in memory via hooks as well as writing them to disk. Then, we could delete local uploads no problem.

I think this is a great feature. We will slate it into our workload. Should not take too much work!

[richardvanbergen]

@jmikrut Agreed. That would be the best solution, I'd rather not have to maintain a separate-but-similar implementation of a resize process that's already happened once in the request.

[richardvanbergen]

Hey @realmacaw just thought you might like to know my plugin is now stable and it has support for disabling the uploads. Please check it out and let me know what you think.

#275

[jmikrut]

@realmacaw quick update for you:

We just released 0.9.2 which now has a super simple way to totally disable writing to the file system!

https://github.com/payloadcms/payload/blob/master/CHANGELOG.md#092-2021-08-06