Special characters in file name throws error

[muizzsiddique]

Specifically the character that can be created on Windows using Alt+0150: '–'. As soon as I rename a file without this the file can be loaded. I tested this both in PyOpenAL and using pyogg.VorbisFile.

This is the error that happens (running in python interactive environment):

>>> pyogg.VorbisFile("50 Cent – Many Men (Wish Death) [Edit].ogg")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\muizz\AppData\Local\Programs\Python\Python38\lib\site-packages\pyogg\__init__.py", line 33, in __init__
raise PyOggError("file couldn't be opened or doesn't exist. Error code : {}".format(error))
pyogg.ogg.PyOggError: file couldn't be opened or doesn't exist. Error code : -1

[mattgwwalker]

Hi Muizz, Thanks for raising this issue. I was afraid that problem might exist. I suspect it exists for all three types: vorbis, opus, and flac. It’s because we’re currently using the file-opening API that the libraries provide. And those APIs dont seem to do Unicode (and you’ve confirmed that). I haven’t looked into it deeply, but I believe the issue could be fixed by passing a file pointer rather than the file name. Are you interested in contributing a solution? If so, I’d be happy to give you a few more suggestions on where to start looking. Cheers, Matthew

…

On Fri, 25 Sep 2020 at 07:21, Muizz Siddique ***@***.***> wrote: Specifically the character that can be created on Windows using Alt+0150: '–'. As soon as I rename a file without this the file can be loaded. I tested this both in PyOpenAL and using pyogg.VorbisFile. This is the error that happens (running in python interactive environment): >>> pyogg.VorbisFile("50 Cent – Many Men (Wish Death) [Edit].ogg") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\muizz\AppData\Local\Programs\Python\Python38\lib\site-packages\pyogg\__init__.py", line 33, in __init__ raise PyOggError("file couldn't be opened or doesn't exist. Error code : {}".format(error)) pyogg.ogg.PyOggError: file couldn't be opened or doesn't exist. Error code : -1 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#53>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA653XBLMAO5A3PAP3MQFXTSHOL4PANCNFSM4RYVX5WA> .

[muizzsiddique]

I think I only have a beginner's grasp on Python/programming but I would love to try and contribute.

Edit: As I thought, this looks far out my scope. I'm still interested but I don't think my abilities are good enough to contribute.

[mattgwwalker]

Ok, that’s good to know. If you have questions then let me know. If you’re keen to contribute, the first step is to fork PyOgg. You’ll need to login to github, find the PyOgg page and click the fork button on the top right. Easy. Now you have your own copy in the cloud. This is a copy of the latest version, which is not the same as the version you might be using if you did a pip install of pyogg. We’re close to creating a new release and it will be great if you can make a contribution to this upcoming version. Next you’ll need a local copy on your computer. Do get this you’ll need Git installed on your computer. If you’re running Mac or Linux, it’s already installed. If your under Windows, you can go a google search and download and install it. With Git installed, you want to do a “git clone” of your fork of PyOgg. This will “connect” your fork of PyOgg with a local copy: you can think of it as a cloud backup if you like. You can get the address to clone from the github page of your fork. You’ll now have a local copy you can work with. You will want to create a virtual environment for your work here. So you’ll need to run something like “python3 -m venv my-pyogg-env” and then depending on if you’re using Mac/Linux or Windows, you’ll need to activate that environment. You can do a google search for this command if you don’t know it; it’s only one line you need to enter. From within your virtual environment you can now install PyOgg in an editable state: “pip install -e .” Next, install pytest: “pip install pytest”. And now you can run the tests included with PyOgg; they should all pass! Just type “pytest” to run them. If you get this far and you’re a beginner, we’ll done. But here’s your challenge: please add a test into test_vorbis_file.py that demonstrates the error You discovered. You may wish to copy and rename the demo vorbis file to do so. Good luck, Matthew You’ve already demonstrated that you can test pyogg to show that there’s an error.

…

On Sat, 26 Sep 2020 at 06:28, Muizz Siddique ***@***.***> wrote: I think I only have a beginner's grasp on Python/programming but I would love to try and contribute. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#53 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA653XGPD525OADXYZNZU7DSHTONJANCNFSM4RYVX5WA> .

[mattgwwalker]

Ah, sorry, I didn't see your edit until after I replied. Feel free to ignore my previous reply, or give it a go, as you wish.

Matthew

[mattgwwalker]

@Zuzu-Typ, I'm looking into fixing this issue. It turns out to be a lot more hairy than I expected. Would you have any comments in this space?

First, we're currently using ov_fopen, which takes a char *. I'm assuming this is the function that isn't Unicode-friendly.

Another option is ov_open, which contains a specific warning for Windows users not to use this function. So we can eliminate this option.

The final option (if we stick with the vorbisfile library) is ov_open_callbacks, which is specifically Windows-friendly. This function requires a FILE pointer.

However, getting a FILE pointer from Python isn't easy. Instead, the recommended approach seems to be to use the C standard library and call fopen using ctypes. However that's tricky given that there seems to be no standard way to obtain access to the C standard library (there is code for Windows but we'd have to specify the library for Mac and Linux differently).

Have you ever done something similar? Do you know of a better way around all this?

[Zuzu-Typ]

Well, it would make no sense to use the fopen function to get a FILE pointer, because that's exactly what ov_fopen does anyways.

I've found this thread on StackOverflow, which recommends the use of GetShortPathName.

I haven't had the time to really test this myself, but I've created a simple implementation which looks promising:

def convert_pathname(pathname):
    buffer = (ctypes.c_wchar * 1024)()
    lpwstr = ctypes.cast(buffer, ctypes.wintypes.LPWSTR)
    ctypes.windll.kernel32.GetShortPathNameW(pathname, lpwstr, 1024)
    return lpwstr.value

And I did try it out for some unicode path name:

>>> convert_pathname("C:/ÄÖÜgaß")
'C:/GA3910~1'

It will only work for existing, full paths though (for the latter os.path.realpath could be used if a relative path is given).
Otherwise it simply returns an empty string.

And obviously it's also limited to about 1000 characters, which could be adjusted in the future if that should be necessary, if paths longer than 256 characters are even supported in the first place.