Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scripts: print the file name when decode syscall #75128

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Sasasu
Copy link

@Sasasu Sasasu commented Jun 27, 2024

the script shows an incorrect file name, I check the wrong file first, then I find the script did not print the current reading file name.

Fix this to prevent others from wasting their time on this.

The new error message:

[1/179] Generating misc/generated/syscalls.json, misc/generated/struct_tags.json
Error decoding zmk/app/modules/hal/altera/drivers/altera_msgdma/HAL/src/altera_msgdma.c (included in zephyr/subsys/net/pkt_filter/ethernet.c)
Traceback (most recent call last):
  File "zephyr/scripts/build/parse_syscalls.py", line 216, in <module>
    main()
  File "zephyr/scripts/build/parse_syscalls.py", line 194, in main
    syscalls, tagged = analyze_headers(args.include, args.scan,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "zephyr/scripts/build/parse_syscalls.py", line 120, in analyze_headers
    contents = fp.read()
               ^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 52638: invalid start byte

@zephyrbot zephyrbot added area: Build System size: XS A PR changing only a single line of code labels Jun 27, 2024
Copy link

Hello @Sasasu, and thank you very much for your first pull request to the Zephyr project!
Our Continuous Integration pipeline will execute a series of checks on your Pull Request commit messages and code, and you are expected to address any failures by updating the PR. Please take a look at our commit message guidelines to find out how to format your commit messages, and at our contribution workflow to understand how to update your Pull Request. If you haven't already, please make sure to review the project's Contributor Expectations and update (by amending and force-pushing the commits) your pull request if necessary.
If you are stuck or need help please join us on Discord and ask your question there. Additionally, you can escalate the review when applicable. 😊

@tejlmand tejlmand added the bug The issue is a bug, or the PR is fixing a bug label Jul 1, 2024
@Sasasu
Copy link
Author

Sasasu commented Jul 2, 2024

https://github.com/zephyrproject-rtos/hal_altera/blame/0d225ddd314379b32355a00fb669eacf911e750d/drivers/altera_msgdma/HAL/src/altera_msgdma.c#L1497C55-L1497C55

I am trying to init a new zyphyr project, looks the non-utf8 code file is not a network error.

/home/sa/W/zmk/app/modules/hal/altera/drivers/altera_msgdma/HAL/src/altera_msgdma.c: line 1497, char 33, byte 52638: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/gigadevice/gd32a50x/standard_peripheral/source/gd32a50x_cmp.c: line 60, char 63, byte 2487: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/gigadevice/gd32a50x/standard_peripheral/source/gd32a50x_rcu.c: line 339, char 38, byte 12482: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/gigadevice/gd32vf103/standard_peripheral/source/gd32vf103_fmc.c: line 42, char 22, byte 1929: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/nuvoton/m2l31x/Devices/M2L31/Include/canfd_reg.h: line 372, char 29, byte 27244: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/nuvoton/m2l31x/Devices/M2L31/Include/usbd_reg.h: line 71, char 153, byte 4369: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/nuvoton/m48x/StdDriver/src/crypto.c: line 1505, char 35, byte 53117: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/nxp/mcux/middleware/mcux-sdk-middleware-usb/output/source/device/class/usb_device_msc.h: line 3, char 25, byte 87: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/nxp/mcux/middleware/wireless/framework_5.3.3/XCVR/MKW40Z4/ifr_mkw40z4_radio.c: line 180, char 62, byte 7084: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_api.h: line 2, char 12, byte 92: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_api_calibration.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_api_core.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_api_ranging.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_api_strings.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_def.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_device.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_interrupt_threshold_settings.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/inc/vl53l0x_tuning.h: line 2, char 11, byte 91: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/src/vl53l0x_api.c: line 2, char 12, byte 92: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/src/vl53l0x_api_calibration.c: line 2, char 12, byte 92: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/src/vl53l0x_api_core.c: line 2, char 12, byte 92: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/src/vl53l0x_api_ranging.c: line 2, char 12, byte 92: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/st/sensor/vl53l0x/api/core/src/vl53l0x_api_strings.c: line 2, char 12, byte 92: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f0xx/drivers/src/stm32f0xx_hal_timebase_rtc_alarm_template.c: line 35, char 53, byte 1753: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f0xx/drivers/src/stm32f0xx_hal_timebase_rtc_wakeup_template.c: line 36, char 53, byte 1778: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f2xx/drivers/src/stm32f2xx_hal_timebase_rtc_alarm_template.c: line 37, char 53, byte 1835: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f2xx/drivers/src/stm32f2xx_hal_timebase_rtc_wakeup_template.c: line 37, char 53, byte 1829: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f3xx/drivers/src/stm32f3xx_hal_timebase_rtc_alarm_template.c: line 35, char 53, byte 1754: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f3xx/drivers/src/stm32f3xx_hal_timebase_rtc_wakeup_template.c: line 36, char 53, byte 1779: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f4xx/drivers/src/stm32f4xx_hal_timebase_rtc_alarm_template.c: line 36, char 53, byte 1787: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32f4xx/drivers/src/stm32f4xx_hal_timebase_rtc_wakeup_template.c: line 37, char 53, byte 1812: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32g0xx/drivers/src/stm32g0xx_hal_timebase_rtc_alarm_template.c: line 36, char 53, byte 1787: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32g0xx/drivers/src/stm32g0xx_hal_timebase_rtc_wakeup_template.c: line 37, char 53, byte 1812: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32g4xx/soc/stm32g4xx.h: line 11, char 48, byte 564: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32h5xx/soc/stm32h5xx.h: line 11, char 48, byte 564: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32l4xx/drivers/src/stm32l4xx_hal_cryp.c: line 111, char 23, byte 6590: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32l4xx/drivers/src/stm32l4xx_hal_mmc.c: line 3432, char 40, byte 119653: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32mp1xx/drivers/include/stm32mp1xx_hal_fdcan.h: line 481, char 65, byte 27248: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32mp1xx/drivers/src/stm32mp1xx_hal_cryp.c: line 458, char 41, byte 22749: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32mp1xx/drivers/src/stm32mp1xx_hal_cryp_ex.c: line 152, char 34, byte 5302: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32mp1xx/drivers/src/stm32mp1xx_hal_wwdg.c: line 47, char 46, byte 2432: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/stm32/stm32cube/stm32mp1xx/soc/stm32mp1xx.h: line 11, char 48, byte 567: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/ti/simplelink/source/ti/devices/cc32xx/inc/hw_aes.h: line 392, char 56, byte 23758: After a first byte between E1 and EC, expecting the 3rd byte between 80 and BF.
/home/sa/W/zmk/app/modules/hal/ti/simplelink/source/ti/devices/cc32xx/inc/hw_common_reg.h: line 420, char 80, byte 20202: After a first byte between E1 and EC, expecting the 3rd byte between 80 and BF.
/home/sa/W/zmk/app/modules/hal/ti/simplelink/source/ti/devices/cc32xx/inc/hw_dthe.h: line 108, char 79, byte 5182: After a first byte between E1 and EC, expecting the 3rd byte between 80 and BF.
/home/sa/W/zmk/app/modules/hal/ti/simplelink/source/ti/devices/cc32xx/inc/hw_hib3p3.h: line 675, char 66, byte 31216: After a first byte between E1 and EC, expecting the 3rd byte between 80 and BF.
/home/sa/W/zmk/app/modules/hal/ti/simplelink/source/ti/drivers/net/wifi/sl_socket.h: line 951, char 73, byte 51922: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/hal/ti/simplelink/source/ti/drivers/net/wifi/wlan.h: line 276, char 9, byte 17173: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/NAMote72/cmsis/stm32l152xc.h: line 12, char 44, byte 631: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/NAMote72/cmsis/stm32l1xx.h: line 11, char 48, byte 576: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/NucleoL152/cmsis/stm32l152xe.h: line 12, char 44, byte 631: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/NucleoL152/cmsis/stm32l1xx.h: line 11, char 48, byte 576: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/NucleoL476/cmsis/stm32l476xx.h: line 10, char 44, byte 458: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/NucleoL476/cmsis/stm32l4xx.h: line 11, char 48, byte 574: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/SKiM880B/cmsis/stm32l151xba.h: line 12, char 44, byte 632: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/SKiM880B/cmsis/stm32l1xx.h: line 13, char 48, byte 624: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/SKiM980A/cmsis/stm32l151xba.h: line 12, char 44, byte 632: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/SKiM980A/cmsis/stm32l1xx.h: line 13, char 48, byte 624: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/mcu/stm32/EEPROM_Emul/Core/eeprom_emul.c: line 1607, char 38, byte 55745: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/mcu/stm32/STM32L4xx_HAL_Driver/Src/stm32l4xx_hal_cryp.c: line 101, char 23, byte 6232: Expecting bytes in the following ranges: 00..7F C2..F4.
/home/sa/W/zmk/app/modules/lib/loramac-node/src/boards/mcu/stm32/STM32L4xx_HAL_Driver/Src/stm32l4xx_hal_swpmi.c: line 308, char 38, byte 14817: Expecting bytes in the following ranges: 00..7F C2..F4.

Can I send PR to those HAL repos? Or try to add more code in this python file to support reading non-utf8 files?

zephyrproject-rtos/hal_altera#2 If this method is acceptable, I can send PR to HAL repo

@tejlmand
Copy link
Collaborator

tejlmand commented Jul 4, 2024

I am trying to init a new zyphyr project, looks the non-utf8 code file is not a network error.

that's a long list of files.
I get curious as to why are all those files suddenly being passed for syscalls ?

Have you made any extra changes in your workspace which triggers this problem ?

Can I send PR to those HAL repos?

Of course, but some repos are maintained outside of Zephyr and then synced regularly, in which case a PR against the origin repo is better.

Or try to add more code in this python file to support reading non-utf8 files?

Before doing so I would like to understand why those files are suddenly being parsed by the parse_syscalls.py, as that doesn't seem to be the case today.

Also see guide-lines here: https://docs.zephyrproject.org/latest/contribute/guidelines.html#coding-style

Avoid using non-ASCII symbols in code, unless it significantly improves clarity, avoid emojis in any case.

and the discussion here: #70610 (comment)

I notice zephyrproject-rtos/hal_altera#2 is actually changing a comment, not code, but I would say utf-8 should provide sufficient support for special chars when there are good reason for such.

@Sasasu
Copy link
Author

Sasasu commented Jul 4, 2024

Have you made any extra changes in your workspace which triggers this problem ?
Or try to add more code in this python file to support reading non-utf8 files?

I run west init inside the app directory, and west.yml also inside in the app directory.
means I set base_path to app/, the OS and HAL code is under base path. then parse_syscalls.py starts to scan every code file under zephyr HAL.

I think modifying the error message is enough, others can find the solution if they do the search in GitHub issues and can find this PR.

The solution is run west init in the first level of git repo, make sure the zephyr directory is the first level.
Example: https://github.com/zephyrproject-rtos/example-application

I notice zephyrproject-rtos/hal_altera#2 is actually changing a comment, not code

Actually, all non-utf8 characters are comment. Like the ' in can't, and some are ®, others look like underline or blod marker

@tejlmand
Copy link
Collaborator

I run west init inside the app directory, and west.yml also inside in the app directory.
means I set base_path to app/, the OS and HAL code is under base path. then parse_syscalls.py starts to scan every code file under zephyr HAL.

Not fully clear to me how your west workspace directory structure looks like based on this description, but this doesn't sound like a T2 or T3 topology, as described here: https://docs.zephyrproject.org/latest/develop/west/workspaces.html#topologies-supported

Note that the example-application uses a T2 topology.

Can you try to sketch your west workspace directory structure using same principle as is used on the supported topology page ?

@kartben
Copy link
Collaborator

kartben commented Aug 9, 2024

@Sasasu could you please address the compliance issues in your commit message. See https://github.com/zephyrproject-rtos/zephyr/actions/runs/9700408884/job/26882772828?pr=75128

@nordicjm
Copy link
Collaborator

@Sasasu best to just remove the log from the commit message so that compliance can pass

the script shows an incorrect file name, I check the wrong file first, then
I find the script did not print the current reading file name.

Fix this to prevent others from wasting their time on this.

The new error message:

```
[1/179] Generating misc/generated/syscalls.json, misc/generated/struct_tags.json
Error decoding zmk/app/modules/hal/altera/drivers/altera_msgdma/HAL/src/altera_msgdma.c (included in zephyr/subsys/net/pkt_filter/ethernet.c)
```

Signed-off-by: Sa Sasu <i@sasa.su>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: Build System bug The issue is a bug, or the PR is fixing a bug size: XS A PR changing only a single line of code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants