This repository has been archived by the owner on Jun 4, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 130
Support images larger than 2GiB in docker_session.upload() #162
Comments
It looks like the root cause of the library failure is the limitation of the SSL wrapped HTTP client - a similar issue discussed at psf/requests#2717 |
When I upload an image larger than 2GiB, I see that the reason is that zlib.crc32() does not support more than (2^32-1) bytes in python version 2.7.5, resulting in an error.zlib.crc32() is called here
$ python2 -c "import zlib;zlib.crc32('a'*(1<<31))"
Traceback (most recent call last):
File "<string>", line 1, in <module>
OverflowError: size does not fit in an int This means that the size of the uploaded image should not exceed 2GiB per layer. We should upgrade python2 to 2.7.18 or python3 |
You're saying this is fixed with just a version bump? |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
This library has been extremely useful for jobs that run docker commands!
One such job recently started failing with
OverflowError: string longer than 2147483647 bytes
when it tried to copy a 2.6GB image.I'm sure that there are reasons behind the 2GiB limit, but would it be possible to increase this limit? (e.g. to 4GiB or 8GiB)
The text was updated successfully, but these errors were encountered: