Compatibility with mozjpeg

[selwin]

Hi there, I was just wondering whether it's possible to use pillow with the newly announced https://github.com/mozilla/mozjpeg/ . Sorry if this is a dumb question

[wiredfool]

Probably, but it might take passing different options in. We already work with libjpeg-turbo, as it's the default on at least one of the linux platforms. As long as they keep the api the same, I don't think it's going to be a major issue to drop it in.

[aclark4life]

Does jpeg2000 support affect this in any way?

[wiredfool]

Nope. Different format.

[blaise-io]

From mozjpeg's readme:

'mozjpeg' is not intended to be a general JPEG library replacement. It makes tradeoffs that are intended to benefit Web use cases and focuses solely on improving encoding. It is best used as part of a Web encoding workflow. For a general JPEG library (e.g. your system libjpeg), especially if you care about decoding, we recommend libjpeg-turbo.

So there may be some caveats even if it's compatible.

[andreymal]

I intalled mozjpeg as default libjpeg in my Arch Linux, and Pillow can't save JPEG files with it:

I/O suspension not supported in scan optimization
Traceback (most recent call last):
  File "./local/genpreview.py", line 33, in go
    cropped_im.save(image_path, quality=75)
OSError: encoder error -2 when writing image file

Pillow still can't work with mozjpeg? Or this is my local problem somewhere?

libjpeg-turbo works good.

[wiredfool]

As far as I know, it's never been tested with mozjpeg.

[rohit-saharan]

+1 , @andreymal its still a problem.

[Bar-Rabbas]

Same here

[strarsis]

Yes, is the quality actually better when the JPEG file is written by mozjpeg instead of passing mozjpeg an existing JPEG file? Only if Pillow can directly write a JPEG using mozjpeg from a raw/lossless buffer this can be realized without intermediary files like PNGs.

[alvassin]

Yep, there is a problem on a OS X. Used following snippet:

from PIL import Image

with open('example.jpg', 'rb') as f:
    image = Image.open(f)
    width, height = image.size
    offer_image = image.crop((0, 0, 20, 20))
    with open('result.jpg') as f:
        offer_image.save(f, "JPEG", quality=90)

On Ubuntu works like a charm. On OS X i see following error when opening JPEG file:

Traceback (most recent call last):
  File "/Users/alvassin/Work/example/t.py", line 46, in <module>
    offer_image = image.crop((0, 0, 20, 20))
  File "/Users/alvassin/Work/example/env/lib/python3.7/site-packages/PIL/Image.py", line 1105, in crop
    self.load()
  File "/Users/alvassin/Work/example/env/lib/python3.7/site-packages/PIL/ImageFile.py", line 270, in load
    raise_ioerror(err_code)
  File "/Users/alvassin/Work/example/env/lib/python3.7/site-packages/PIL/ImageFile.py", line 59, in raise_ioerror
    raise OSError(message + " when reading image file")
OSError: broken data stream when reading image file

If i open PNG and convert it to RGB, i can't save it as JPEG:

from PIL import Image

with open('example.png', 'rb') as f:
    image = Image.open(f)
    image = image.convert('RGB')
    width, height = image.size
    offer_image = image.crop((0, 0, 20, 20))
    with open('result.jpg') as f:
        offer_image.save(f, "JPEG", quality=90)

Error:

Wrong JPEG library version: library is 90, caller expects 80
Traceback (most recent call last):
  File "/Users/alvassin/Work/example/env/lib/python3.7/site-packages/PIL/ImageFile.py", line 496, in _save
    fh = fp.fileno()
io.UnsupportedOperation: fileno

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/alvassin/Work/example/t.py", line 20, in <module>
    offer_image.save(f, "JPEG", quality=90)
  File "/Users/alvassin/Work/example/env/lib/python3.7/site-packages/PIL/Image.py", line 2102, in save
    save_handler(self, fp, filename)
  File "/Users/alvassin/Work/example/env/lib/python3.7/site-packages/PIL/JpegImagePlugin.py", line 766, in _save
    ImageFile._save(im, fp, [("jpeg", (0, 0) + im.size, 0, rawmode)], bufsize)
  File "/Users/alvassin/Work/example/env/lib/python3.7/site-packages/PIL/ImageFile.py", line 515, in _save
    raise OSError("encoder error %d when writing image file" % s)
OSError: encoder error -2 when writing image file

How is it possible to fix the issue? Could somebody please give an advice where to dig further?

[Bar-Rabbas]

OSError: broken data stream when reading image file

Do you get this error with any image file? Seems to me like a broken jpeg file or maybe a broken connection during file opening on a remote filesystem or something like this. Try with other files first.

Wrong JPEG library version: library is 90, caller expects 80

Here you have an updated jpeg library (version 90) and pil expects 80. Probably you should upgrade all modules.

[alvassin]

@DonnaRosa Thank very much for your response!

Do you get this error with any image file? Seems to me like a broken jpeg file or maybe a broken connection during file opening on a remote filesystem or something like this. Try with other files first.

I've tried several files. For example, that one. They are opened and saved correctly using jpeg, jpeg-turbo (last jpeg-turbo test failed, hovewer previously it was working well, need to investigate that) and openjpeg libraries. But problem with mozjpeg remains on OS X.

Here you have an updated jpeg library (version 90) and pil expects 80. Probably you should upgrade all modules.

How could i do that? I am trying to install latest Pillow/Pillow-SIMD versions (tried both)

UPD:

jpeg: OK
export CFLAGS="-I$(brew --prefix zlib)/include  -I$(brew --prefix jpeg)/include"
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow: OK
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow-SIMD: OK

openjpeg: OK
export CFLAGS="-I$(brew --prefix zlib)/include  -I$(brew --prefix openjpeg)/include"
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow: OK
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow-SIMD: OK

jpeg-turbo: OSError: broken data stream when reading image file
export CFLAGS="-I$(brew --prefix zlib)/include  -I$(brew --prefix jpeg-turbo)/include"
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow: NOT OK
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow-SIMD: NOT OK

mozjpeg: OSError: broken data stream when reading image file
export CFLAGS="-I$(brew --prefix zlib)/include  -I$(brew --prefix mozjpeg)/include"
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow: NOT OK
pip install --no-binary :all: --no-cache-dir --force-reinstall -U Pillow-SIMD: NOT OK

[petrprikryl]

@DonnaRosa Thank very much for your response!

I've tried several files. For example, that one. They are opened and saved correctly using jpeg, jpeg-turbo (last jpeg-turbo test failed, hovewer previously it was working well, need to investigate that) and openjpeg libraries. But problem with mozjpeg remains on OS X.

I have tried to open your example image. But Ubuntu's EyeOfGnome couldn't open it:
Not a JPEG file: starts with 0x52 0x49. And based on this error message it seems that it is WebP image actually https://askubuntu.com/a/1029471/501428.

[radarhere]

The image from the last few comments no longer seems available for download.

Apart from that, I've created #8645 as a possible resolution to this.