-
Notifications
You must be signed in to change notification settings - Fork 346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix functionality on Windows #820
Open
jktjkt
wants to merge
3
commits into
mbj4668:master
Choose a base branch
from
jktjkt:windows
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The current approach is not recoomended [1], and it has not worked on Windows since the distribution switched to Python wheels. With wheels, no code from `setup.py` runs at the installation time, which means that platform detection logic which tried to prepare BAT files on Windows was non-effective. The wheels are marked using the `any` tag which means "pure Python code only", which is a correct marking, but then the actual content of wheels would differ depending on whether they were *built* on Windows vs. Unix. The modern alternative is to let `setuptools` create these scripts automatically [2]. That thing apparently puts, e.g., `pyang` into $PATH on Windows, possibly generating all the requires wrappers and what not. That sounds like a best thing since sliced bread, and indeed it is. Discovered via GNPy, thanks to Stefan Melin from Telia for reporting a test failure on Windows. [1] https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#scripts [2] https://setuptools.pypa.io/en/latest/userguide/quickstart.html#entry-points-and-automatic-script-creation
This code attempted to produce Python wrappers around native Python code and creating some `.bat` files on Windows. The previous commit ported all Python scripts to a native equivalent from setuptools. Since the only remaining script is a bash one, not a Python code, let's remove that special casing. Those on Windows need Bash for this script anyway, so let them invoke `bash path/to/yang2dsdl` directly.
The code was trying to be portable by using the OS-specific path separator in a regular expression. However, on Windows the path separator is a backslash which is a special character in a regular expression. However, simply wrapping `os.pathsep` in `re.compile()` would not be the right thing to do here. It is very common to also use paths with Unix-style forward slashes on Windows, especially with portable projects which want to use the same scripts on Unix. Since forward slashes are not allowed in file names on Windows (and they can be used "instead of" backslashes in a lot of contexts), using both is OK on Windows. Anyone using backslashes in, say, Linux, will see a change of behavior, but come on, that would not exactly be the sanest thing to do. Also, YANG disallows a lot of funny characters in module names, so let's be reasonable here. Of course the best fix here would be to use something like pathlib for path handling, and only apply the regex on actual file names, but I would prefer to use pyang in Windows CI for our project without doing a major refactoring here. Fixes: dad5c68 Fix issue mbj4668#809: revision-date parsed wrong if >1 @ in path
jktjkt
changed the title
Fix script functionality on Windows
Fix functionality on Windows
Jul 7, 2022
For the "Fix invalid regular expression on Windows" part, I fixed it by |
As I just got informed, @jktjkt only cares about the invalid regular expression on windows which is fixed now. I am interested in moving the scrips into the repository and using entry_points. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've started using
pyang
in a test suite of our project for linting the YANG files. Our test suite simply tried to executepyang something something
, which failed on Windows. Upon investigating, it turned out that Wheel-based installation won't include the.bat
files, and upon investigating this further, it seems that the code tried to wrap Bash shell script with a Python shebang. Hence:scripts: port to entry_points.console_scripts
The current approach is not recoomended, and it has not worked on Windows since the distribution switched to Python wheels. With wheels, no code from
setup.py
runs at the installation time, which means that platform detection logic which tried to prepare BAT files on Windows was non-effective. The wheels are marked using theany
tag which means "pure Python code only", which is a correct marking, but then the actual content of wheels would differ depending on whether they were built on Windows vs. Unix.The modern alternative is to let
setuptools
create these scripts automatically. That thing apparently puts, e.g.,pyang
into$PATH
on Windows, possibly generating all the requires wrappers and what not. That sounds like a best thing since sliced bread, and indeed it is.Discovered via GNPy, thanks to Stefan Melin from Telia (@Sme791) for reporting a test failure on Windows.
Do not wrap a shell script with Python on Windows
This code attempted to produce Python wrappers around native Python code and creating some
.bat
files on Windows. The previous commit ported all Python scripts to a native equivalent from setuptools.Since the only remaining script is a bash one, not a Python code, let's remove that special casing. Those on Windows need Bash for this script anyway, so let them invoke
bash path/to/yang2dsdl
directly.Fix invalid regular expression on Windows
The code (Cc: @mlittlej, #809) was trying to be portable by using the OS-specific path separator in a regular expression. However, on Windows the path separator is a backslash which is a special character in a regular expression.
However, simply wrapping
os.pathsep
inre.compile()
would not be the right thing to do here. It is very common to also use paths with Unix-style forward slashes on Windows, especially with portable projects which want to use the same scripts on Unix. Since forward slashes are not allowed in file names on Windows (and they can be used "instead of" backslashes in a lot of contexts), using both is OK on Windows. Anyone using backslashes in, say, Linux, will see a change of behavior, but come on, that would not exactly be the sanest thing to do. Also, YANG disallows a lot of funny characters in module names, so let's be reasonable here.Of course the best fix here would be to use something like pathlib for path handling, and only apply the regex on actual file names, but I would prefer to use pyang in Windows CI for our project without doing a major refactoring here.