Encoding problems when body contains non ascii chars

[lucasbrunialti]

First, j0057 that's a really nice project! :)

Using Python2.7 and running the following command raises an exception:

githubrelease release Cobliteam/ommited edit v1.0.1-dev.3 --body 'Merges since 2018-01-24 18:38:46 +0000: - message with non ascii char: … '

Exception traceback:

Traceback (most recent call last):
  File "/home/travis/build/Cobliteam/ommited/venv/bin/githubrelease", line 11, in <module>
    sys.exit(main())
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/github_release.py", line 482, in _cli_release_edit
    gh_release_edit(*args, **kwargs)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/github_release.py", line 116, in with_check_for_credentials
    return func(*args, **kwargs)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/github_release.py", line 497, in gh_release_edit
    patch_release(repo_name, current_tag_name, **attributes)
  File "/home/travis/build/Cobliteam/ommited/venv/local/lib/python2.7/site-packages/github_release.py", line 362, in patch_release
    current_tag_name, "\n  ".join(updated)))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2026' in position 167: ordinal not in range(128)

[lucasbrunialti]

I think a simple # -*- coding: utf-8 -*- in the header solve the problem, I'm trying a fix, can you predict any problem that this can cause?

Also, I'm having trouble running tests locally, the function do_commit can not find the VERSION file, so it fails many tests. I can make a fix for that too, but I don't know if I've just missed some setup...

Thank you.

[jcfr]

Hi Lucas, Thanks for the kind words and the patch. Out until Tuesday morning (and away from computer), I will be able to have a look only later. Thanks Jc On Jan 26, 2018 4:27 PM, "lucasbrunialti" <notifications@github.com> wrote: I think a simple # -*- coding: utf-8 -*- in the header solve the problem, I'm trying a fix, can you predict any problem that this can cause? Also, I'm having trouble running tests locally, the function do_commit can not find the VERSION file, so it fails many tests. I can make a fix for that too, but I don't if I've just missed some setup... Thank you. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#40 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANXo_hzLaTHLjitEBerRflOs_In-a_7ks5tOm2KgaJpZM4RvDbG> .

[lucasbrunialti]

We also had to set PYTHONIOENCODING=utf8 (we also use travis)...

[jcfr]

Trying to reproduce the encoding issue locally, I tried the following:

$ mkvirtualenv test-issue-40

$ python --version 
Python 2.7.10

$ pip install githubrelease
$ githubrelease release jcfr/sandbox edit 1.0.0 --body 'This is a message with non ascii chars: …❄'
updating '1.0.0' release: 
  body: None -> 'This is a message with non ascii chars: …❄'

$ python -c "import locale; print(locale.getpreferredencoding())"
UTF-8

but this doesn't trigger the error

[lucasbrunialti]

I'm thinking the problem is not with the github-release itself, but with python's io encoding... if sys.stdout.encoding is None, then a simple print of a utf8 encoding string will raise an exception, because the default sysout is ascii encoding.
I just don't know if /pull/43 fix a bug that we could not find...

[jcfr]

Could this be related to http://bugs.python.org/issue19846

And here is a quote from https://bugs.python.org/issue19846#msg205545

At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.