Running without pillow

[tykling]

Hello,

Is it possible to run this without pillow at all if all the processing is done async/remotely on another server?

Thanks! :)

[tykling]

To elaborate a bit: I would love to use all the cool work you did on rendering <picture> elements, like taking breakpoints and aspect ratios and multiple formats and all that into account.

But I need a "lite" version which only does the bit in utils.py and models.py and never loads pillow. I can fork it of course, or just borrow some bits from django-pictures, but I wanted to ask if you see a way to do what I want with the current django-pictures?

Thanks! 👍

[codingjoe]

Hi @tykling,

Yes, that “should” be possible. I designed the package to be able to support this use case. I believe multiple platforms might mature into that direction.

All that being said, I have not put it to a test personally yet. Soooo, we might have to figure this out together and while doing so, maybe write a bit more documentation.

I would certainly be committed to implement the necessary changes to make this work, should it be needed.

Perhaps you can start by giving me a run-down on how you currently serve and process images? And we take it from there?

Cheers!
Joe

[tykling]

Thank you for your response, that sounds promising! 👍

I am building a gallery of sorts which will serve pictures and other files. The part serving the pictures is currently just an <img> tag but I want to serve different sizes, aspect ratios, etc. and django-pictures looks perfect for that.

I would love to have a way to use a PictureField in a way where I provide all the files and fill the model fields and stuff it needs so I can use it in templates etc. like in your docs but without actually doing any of the processing on the server.

In my project the file uploads, and subsequent resizing of pictures, exif extraction and other handling of binary files are all handled by remote clients via an HTTP API. It has endpoints to upload the original file, and then resized versions, exit metadata etc. Files are eventually served by nginx using x-accel-redirect and django doesn't ever really handle them after the upload. This is by design.

So, the flow would be:

1. Client uses API to upload a file which is saved to the PictureField. Some new setting prevents django-pictures from doing anything apart from saving the original file.
2. In the API response the client is informed that for the uploaded file the system needs some more work done
3. Client does all the requested resizing, uploads the results using the API, and the API knows where to put the files with the right names
4. Client continues with the next file

Hm, thinking about it, could I get all of this by pre-filling width and height fields with values provided by the client with the initial upload, and then configuring django-pictures to use a message queue which doesn't do anything? Then the only change I would need would be to make pillow optional?

[tykling]

Ok, I've made some progress on this.

Because PictureField inherits from Djangos ImageField and both import from PIL I've had to make a small shim to make the imports not fail. I also configured django-pictures with a dummy/null processor. Together this means I can use django-pictures like I want without pillow installed at all.

I am not sure if making pillow an optional dep in django-pictures would make this simpler, because Djangos own ImageField also imports pillow. I might open up an issue with Django itself and see if we can make the pillow-importing parts of ImageField optional somehow, but until that happens I see no reason to change anything in django-pictures.

It'll be a few days before I get back to this but I will update here once I get this all committed and pushed to Github. For now I'll include the fake PIL I'm using:

$ cat PIL/__init__.py 
"""Fake PIL to make imports in ImageField and PictureField work."""

class Image:
    pass

class ImageOps:
    pass

class ImageDraw:
    pass

class ImageFont:
    pass

[codingjoe]

Hi @tykling,

Thank you for your patience. It's been a busy year. However, I really wanted to get back on this topic.

I believe this requires 2 steps:

1. Disabling the native image processor.
   This part is rather simple. You only have to set the PROCESSOR setting to a function that does nothing.
2. Getting the correct URLs.
   This part isn't too difficult either, but might benefit from a small patch. You will need to override the SimplePicture.url property. We already alter the behavior when USE_PLACEHOLDERS is active.

I will draft a patch, including documentation on how to set up the package with an external image processor. I'd love if you can give it a review and see if it meets your requirements.

Cheers!
Joe

[codingjoe]

Hm… I can decouple Pillow from this package, though I wouldn't document it. This seems like a rather odd use case. Can you tell me why you don't want to install Pillow?

[codingjoe]

Also, a small warning. Django does not consider Pillow an optional dependency when using an ImageField. The code might change in any release without an additional warning.

[tykling]

Mainly because it has a history of many CVEs and I really don't want to create a new release of my thing for vulns in a dep I don't even use.

Processing binary files directly on the server is an infosec nightmare, which is why I started this journey to begin with. Even just reading the dimensions of a malicious image with pillow can be enough to trigger an RCE. Removing pillow from the equation entirely seems like the best option, in case some future django update starts using it in some new ways.

Making Pillow an optional dependency in django-pictures would be a hassle for most users so that is out of the question of course. Sadly it isn't possible to substract deps from the main dependencies in extras, otherwise having a django-pictures[nopillow] extra defined in pyproject.toml might be a solution.

[codingjoe]

Hm… OK, how do you get the with and height into the width and height fields? I could create a version that doesn't require Pillow during import at all, but I wouldn't document this behavior. For the majority of users, this seems to be a rather sophisticated setup.

[tykling]

Width and height and exif and other interesting values are extracted from the image by a worker on a different server and delivered to django via an API endpoint. When I create an instance of the model with the PictureField I just provide width and height like any other model field value.

I absolutely agree that this is super niche, and I appreciate you even taking the time to discuss it, thank you.