Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix phase2 failures and improve QA #26

Merged
merged 7 commits into from
Apr 13, 2024
Merged

Fix phase2 failures and improve QA #26

merged 7 commits into from
Apr 13, 2024

Commits on Apr 13, 2024

  1. phase2: dlsha2rsyncpl: fix sha2rsync.pl script location 

    Fixes following error:

 File 'sha2rsync.pl' not available at master

Fixes: c3ddb0d ("phase2: use sha2rsync.pl for 'targetupload'")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
    @ynezz
    ynezz committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    cff3aab View commit details
    Browse the repository at this point in the history

  2. phase2: fix stray closing parenthesis 

    Due to the stray closing parenthesis introduced in commit f0faed2
("phase2: compute checksums") we have currently we've failing checksum
step in phase2:

 argv: b'cd bin/packages/mipsel_24kc_24kf; find . -type f -not -name \'sha256sums\' -printf "%P\n" | sort | xargs -r ../../../staging_dir/host/bin/mkhash -n sha256 | sed -ne \'s!^\\(.*\\) \\(.*\\)$!\x01 *\x02!p\' > sha256sums)'
 environment:
  ...snip...
  using PTY: False
 /bin/sh: 2: Syntax error: ")" unexpected
 program finished with exit code 2

Fixes: f0faed2 ("phase2: compute checksums")
Reported-by: Hannu Nyman <hannu.nyman@iki.fi>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
    @ynezz
    ynezz committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    56f1ba5 View commit details
    Browse the repository at this point in the history

  3. Add linting with flake8 tool 

    flake8 is a Python tool that glues together pycodestyle, pyflakes,
mccabe, and third-party plugins to check the style and quality of some
Python code.

Currently we've issue in phase2 in checksum step:

 /bin/sh: 2: Syntax error: ")" unexpected

And it seems, that flake8 is able to spot places which might lead to
such issues during runtime:

 phase2/master.cfg:733:151: W605 invalid escape sequence '\('
 phase2/master.cfg:733:155: W605 invalid escape sequence '\)'
 phase2/master.cfg:733:158: W605 invalid escape sequence '\('
 phase2/master.cfg:733:162: W605 invalid escape sequence '\)'

So lets enable flake8 checking on the CI so we can spot similar places
in the future and address them before deployment.

We dont want to make current ongoing work on phase2 code harder, thus we
don't touch that part yet, so we whitelist most of the checks for now.

References: f0faed2 ("phase2: compute checksums")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
    @ynezz
    ynezz committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    e99b029 View commit details
    Browse the repository at this point in the history

  4. phase2: checksums: split long command line 

    To make it more readable, diffs smaller, easier to review etc.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
    @ynezz
    ynezz committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    0b70ae6 View commit details
    Browse the repository at this point in the history

  5. phase2: fix invalid escape sequence flake8 warning in checksums step 

    Resolved a flake8 linting warning related to an invalid escape sequence
in the ShellCommand for calculating checksums:

 phase2/master.cfg:739:28: W605 invalid escape sequence '\('
 phase2/master.cfg:739:32: W605 invalid escape sequence '\)'
 phase2/master.cfg:739:35: W605 invalid escape sequence '\('
 phase2/master.cfg:739:39: W605 invalid escape sequence '\)'

The warning was caused by the use of unescaped parentheses in a regular
expression within a sed command.

Use a raw string (with an 'r' prefix) to treat backslashes as literal
characters, ensuring that the regular expression is correctly
interpreted and flake8 does not raise a warning.

This fix ensures that the code adheres to Python's string handling best
practices and maintains the integrity of the regular expression
functionality.

Fixes: f0faed2 ("phase2: compute checksums")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
    @ynezz
    ynezz committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    0f42b18 View commit details
    Browse the repository at this point in the history

  6. requirements-dev: use black 23.12.1 and ruff 0.1.9 

    Fix ruff's recommendation:

 phase2/master.cfg:328:16: E721 Do not compare types, use `isinstance()`

 "Unlike a direct type comparison, isinstance will also check if an
  object is an instance of a class or a subclass thereof."

Signed-off-by: Petr Štetiar <ynezz@true.cz>
    @ynezz
    ynezz committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    8686448 View commit details
    Browse the repository at this point in the history

  7. phase2: fix relative paths for scripts and files 

    Currently `buildlist` step fails with following:

  ../../../sha2rsync.pl ../../arch-sha256sums bin/packages/aarch64_generic/sha256sums rsynclist
   in dir /builder/aarch64_generic/build/sdk (timeout 1200 secs)
   watching logfiles {}
   argv: [b'../../../sha2rsync.pl', b'../../arch-sha256sums', b'bin/packages/aarch64_generic/sha256sums', b'rsynclist']
   environment:
    ...
    PWD=/builder/aarch64_generic/build/sdk
    ...
  Upon execvpe b'../../../sha2rsync.pl' [b'../../../sha2rsync.pl', b'../../arch-sha256sums', b'bin/packages/aarch64_generic/sha256sums', b'rsynclist'] in environment id 139847367136832
  :Traceback (most recent call last):
  ...
  FileNotFoundError: [Errno 2] No such file or directory: b'../../../sha2rsync.pl'

due to relative paths being off by one:

 worker work dir = /builder/aarch64_cortex-a72/build
 workerdest = "../sha2rsync.pl"
   is absolute path /builder/aarch64_cortex-a72/sha2rsync.pl

thus relative path from:

  FileNotFoundError: [Errno 2] No such file or directory: b'../../../sha2rsync.pl'

in following context:

  PWD=/builder/aarch64_generic/build/sdk
  b'../../../sha2rsync.pl'

is wrong absolute path `/builder/sha2rsync.pl` by one directory level,
thus adjust all those paths accordingly.

Fixes: c3ddb0d ("phase2: use sha2rsync.pl for 'targetupload'")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
    @ynezz
    ynezz committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    fe54cb8 View commit details
    Browse the repository at this point in the history