Add compatMode raw JSON output and fix tls_verify init on pull()

[D3vil0p3r]

Sorry @rhatdan This PR replaces #495 due to some conflicts and need of rebase. For simplicity I opened this one in a clean manner.

Fix #492

Added compatMode param to pull() function. Default as True in order to give a detailed progress JSON output by default like:

...
{'status': 'Pulling fs layer', 'progressDetail': {}, 'id': 'ef3afbc03436'}
{'status': 'Downloading', 'progressDetail': {'current': 1862903, 'total': 37308955}, 'progress': '[==>                                                ]  1.863MB/37.31MB', 'id': '1546de61b6c3'}
{'status': 'Download complete', 'progressDetail': {}, 'id': 'fa424c11eb04'}
{'status': 'Pulling fs layer', 'progressDetail': {}, 'id': '3d3d38ab8766'}
{'status': 'Downloading', 'progressDetail': {'current': 1441042, 'total': 3248294}, 'progress': '[======================>                            ]  1.441MB/3.248MB', 'id': 'ef3afbc03436'}
...

while compatMode=False still keeps the one-field stream JSON output:

{'stream': 'Copying blob sha256:30babffc8f090f7c78c263b9a1b683b34ca5f73da74911ffe942d4d8225dca57\n'}

This approach differs from the already present progress_bar=True because does not show the "fancy" progress bar but a raw detailed progressing JSON output, useful for devs want to manipulate pull output.

Furthermore, the PR fixes tls_verify initialization in pull() because, according to its description, its Default should be True, but it was actually as None.

[rhatdan]

Please squash your commits.

[D3vil0p3r]

I investigated why "Test on Fedora" is failing. It is due to the different image ID that is obtained when used compatMode=True or compatMode=False, where, in the first case, it causes ImageNotFound during pytest tests. Indeed, by directly invoking the API with compatMode=false:

curl -XPOST --unix-socket /run/user/1000/podman/podman.sock -H content-type:application/json "http://d/v4.0.0/libpod/images/pull?reference=quay.io/libpod/alpine&compatMode=false"

produces:

{"stream":"Getting image source signatures\n"}
{"stream":"Copying blob sha256:9d16cba9fb961d1aafec9542f2bf7cb64acfc55245f9e4eb5abecd4cdc38d749\n"}
{"stream":"Copying config sha256:961769676411f082461f9ef46626dd7a2d1e2b2a38e6a44364bcbecf51e66dd4\n"}
{"stream":"Writing manifest to image destination\n"}
{"images":["961769676411f082461f9ef46626dd7a2d1e2b2a38e6a44364bcbecf51e66dd4"],"id":"961769676411f082461f9ef46626dd7a2d1e2b2a38e6a44364bcbecf51e66dd4"}

and the obj["id"] value in

podman-py/podman/domain/images_manager.py

Line 421 in 8cab9d8

return self.get(obj["id"])

will correctly be:

961769676411f082461f9ef46626dd7a2d1e2b2a38e6a44364bcbecf51e66dd4

By directly invoking the API with compatMode=true

curl -XPOST --unix-socket /run/user/1000/podman/podman.sock -H content-type:application/json "http://d/v4.0.0/libpod/images/pull?reference=quay.io/libpod/alpine&compatMode=true"

produces:

{"status":"Pulling fs layer","progressDetail":{},"id":"9d16cba9fb96"}
{"status":"Download complete","progressDetail":{},"id":"9d16cba9fb96"}
{"status":"Pulling fs layer","progressDetail":{},"id":"961769676411"}
{"status":"Download complete","progressDetail":{},"id":"961769676411"}
{"status":"Download complete","progressDetail":{},"id":"961769676411"}

BUT the obj["id"] value in

podman-py/podman/domain/images_manager.py

Line 421 in 8cab9d8

return self.get(obj["id"])

will wrongly be:

9d16cba9fb96

This ID will cause ImageNotFound error during the pytest tests.

It seems that response in pull() function, will consider only the first output line above, that contains the wrong ID field:

...
for item in response.iter_lines():
    obj = json.loads(item)
    print(obj)

{'status': 'Pulling fs layer', 'progressDetail': {}, 'id': '9d16cba9fb96'}

It should refer to the last line instead of first. To solve this, I deal the for item in response.iter_lines(): as for item in reversed(list(response.iter_lines())):. By referring to the last item, it will guarantee that on both the scenarios (compatMode true or false) work correctly, it will retrieve the correct image ID.

Indeed now "Test on Fedora" works.

[D3vil0p3r]

Please squash your commits.

Commits squashed.

[inknos]

/lgtm
/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3vil0p3r, inknos

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [inknos]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment