-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
To fix the issue: show_techsupport & saidump errors during testbed testing by replacing redis-rdb-tool with rdb-cli #19268
Conversation
@mlok-nokia @kcudnik, please help to review, thanks. |
@qiluo-msft, could you please review and approve this PR? Thanks. |
@qiluo-msft, could you please review and approve this PR? Thanks. |
@qiluo-msft reminder to review/approve. |
Hello, @qiluo-msft reminder to review/approve. |
Hello, @qiluo-msft. Please help to review and approve. It's been a long time since this PR was created. |
@qiluo-msft @lguohan. Please help to review and approve. Thanks. |
…sting by replacing redis-rdb-tool with rdb-cli (#1391) Why I did it Fix issue: #1387 The latest redis-rdb-tools-0.1.15 doesn't support Redis 7.0. Redis 7.0 was released in 2020 and adopted by SONiC's latest version. So, this issue turned out. https://github.com/sripathikrishnan/redis-rdb-tools I.e., the rdb-tools is far behind the Redis 7.0. The librdb can perfectly fix this issue. Please see quote from https://github.com/redis/librdb. Motivation behind this project There is a genuine need by the Redis community for a versatile RDB file parser that can export data, perform data analysis, or merely extract raw data from RDB and RESTORE it against a live Redis server. However, available parsers have shortcomings in some aspects such as lack of long-term support, lagging far behind the latest Redis release, and usually not being optimized for memory, performance, or high-traffic streaming for production environments. Additionally, most of them are not written in C, which limits the reuse of Redis components and the potential to contribute back to Redis repo. To address these issues, it is worthwhile to develop a new parser with a modern architecture, that maybe can also challenge the current integrated RDB parser of Redis and even replace it in the future. So, the below PRS are to replace rdbtools with librdb's tool rdb-cli. sonic-net/sonic-buildimage#19268 #1391
8a4969a
to
63ad511
Compare
@qiluo-msft : can you please merge this |
@lguohan @xumia @qiluo-msft Please help to merge this PR since related PR has been merge already |
…sting by replacing redis-rdb-tool with rdb-cli (sonic-net#19268) * To fix the issue: show_techsupport & saidump errors during testbed testing sonic-net/sonic-sairedis#1387 To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage repo's platform/broadcom/docker-syncd-brcm-dnx/Dockerfile.j2, install rdb-cli into the syncd containter. (2) Updated sonic-sairedis repo's script file: files/scripts/saidump.sh, replace rdbtools with rdb-cli. (3) Updated sonic-sairedis repo's saidump/saidump.cpp, to process the rdb-cli's ouput json file.
Cherry-pick PR to 202405: #20728 |
…sting by replacing redis-rdb-tool with rdb-cli (#19268) * To fix the issue: show_techsupport & saidump errors during testbed testing sonic-net/sonic-sairedis#1387 To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage repo's platform/broadcom/docker-syncd-brcm-dnx/Dockerfile.j2, install rdb-cli into the syncd containter. (2) Updated sonic-sairedis repo's script file: files/scripts/saidump.sh, replace rdbtools with rdb-cli. (3) Updated sonic-sairedis repo's saidump/saidump.cpp, to process the rdb-cli's ouput json file.
@JunhongMao , @qiluo-msft , this PR breaks broadcom build.
|
I have created a new PR below to fix it. Please help to review it. Thanks. |
…sting by replacing redis-rdb-tool with rdb-cli (sonic-net#19268) * To fix the issue: show_techsupport & saidump errors during testbed testing sonic-net/sonic-sairedis#1387 To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage repo's platform/broadcom/docker-syncd-brcm-dnx/Dockerfile.j2, install rdb-cli into the syncd containter. (2) Updated sonic-sairedis repo's script file: files/scripts/saidump.sh, replace rdbtools with rdb-cli. (3) Updated sonic-sairedis repo's saidump/saidump.cpp, to process the rdb-cli's ouput json file.
…sting by replacing redis-rdb-tool with rdb-cli (sonic-net#1391) Why I did it Fix issue: sonic-net#1387 The latest redis-rdb-tools-0.1.15 doesn't support Redis 7.0. Redis 7.0 was released in 2020 and adopted by SONiC's latest version. So, this issue turned out. https://github.com/sripathikrishnan/redis-rdb-tools I.e., the rdb-tools is far behind the Redis 7.0. The librdb can perfectly fix this issue. Please see quote from https://github.com/redis/librdb. Motivation behind this project There is a genuine need by the Redis community for a versatile RDB file parser that can export data, perform data analysis, or merely extract raw data from RDB and RESTORE it against a live Redis server. However, available parsers have shortcomings in some aspects such as lack of long-term support, lagging far behind the latest Redis release, and usually not being optimized for memory, performance, or high-traffic streaming for production environments. Additionally, most of them are not written in C, which limits the reuse of Redis components and the potential to contribute back to Redis repo. To address these issues, it is worthwhile to develop a new parser with a modern architecture, that maybe can also challenge the current integrated RDB parser of Redis and even replace it in the future. So, the below PRS are to replace rdbtools with librdb's tool rdb-cli. sonic-net/sonic-buildimage#19268
…sting by replacing redis-rdb-tool with rdb-cli (#1391) (#1471) Why I did it Fix issue: #1387 The latest redis-rdb-tools-0.1.15 doesn't support Redis 7.0. Redis 7.0 was released in 2020 and adopted by SONiC's latest version. So, this issue turned out. https://github.com/sripathikrishnan/redis-rdb-tools I.e., the rdb-tools is far behind the Redis 7.0. The librdb can perfectly fix this issue. Please see quote from https://github.com/redis/librdb. Motivation behind this project There is a genuine need by the Redis community for a versatile RDB file parser that can export data, perform data analysis, or merely extract raw data from RDB and RESTORE it against a live Redis server. However, available parsers have shortcomings in some aspects such as lack of long-term support, lagging far behind the latest Redis release, and usually not being optimized for memory, performance, or high-traffic streaming for production environments. Additionally, most of them are not written in C, which limits the reuse of Redis components and the potential to contribute back to Redis repo. To address these issues, it is worthwhile to develop a new parser with a modern architecture, that maybe can also challenge the current integrated RDB parser of Redis and even replace it in the future. So, the below PRS are to replace rdbtools with librdb's tool rdb-cli. sonic-net/sonic-buildimage#19268 Co-authored-by: JunhongMao <134556118+JunhongMao@users.noreply.github.com>
This was missed on the original PR and caused show_techsupport sonic-mgmt tests to fail. Why I did it PR #19268 replaced redis-rdb-tool with rdb-cli in the Broadcom XGS syncd container. However, it did not do the same for the DNX syncd container, leading to sonic-mgmt failures in show_techsupport tests on DNX platforms. How I did it I applied the same change to the Broadcom DNX syncd container. How to verify it Run sonic-mgmt show_techsupport tests on a Broadcom DNX DUT.
This was missed on the original PR and caused show_techsupport sonic-mgmt tests to fail. Why I did it PR sonic-net#19268 replaced redis-rdb-tool with rdb-cli in the Broadcom XGS syncd container. However, it did not do the same for the DNX syncd container, leading to sonic-mgmt failures in show_techsupport tests on DNX platforms. How I did it I applied the same change to the Broadcom DNX syncd container. How to verify it Run sonic-mgmt show_techsupport tests on a Broadcom DNX DUT.
This was missed on the original PR and caused show_techsupport sonic-mgmt tests to fail. Why I did it PR #19268 replaced redis-rdb-tool with rdb-cli in the Broadcom XGS syncd container. However, it did not do the same for the DNX syncd container, leading to sonic-mgmt failures in show_techsupport tests on DNX platforms. How I did it I applied the same change to the Broadcom DNX syncd container. How to verify it Run sonic-mgmt show_techsupport tests on a Broadcom DNX DUT.
…sting by replacing redis-rdb-tool with rdb-cli (sonic-net#1391) Why I did it Fix issue: sonic-net#1387 The latest redis-rdb-tools-0.1.15 doesn't support Redis 7.0. Redis 7.0 was released in 2020 and adopted by SONiC's latest version. So, this issue turned out. https://github.com/sripathikrishnan/redis-rdb-tools I.e., the rdb-tools is far behind the Redis 7.0. The librdb can perfectly fix this issue. Please see quote from https://github.com/redis/librdb. Motivation behind this project There is a genuine need by the Redis community for a versatile RDB file parser that can export data, perform data analysis, or merely extract raw data from RDB and RESTORE it against a live Redis server. However, available parsers have shortcomings in some aspects such as lack of long-term support, lagging far behind the latest Redis release, and usually not being optimized for memory, performance, or high-traffic streaming for production environments. Additionally, most of them are not written in C, which limits the reuse of Redis components and the potential to contribute back to Redis repo. To address these issues, it is worthwhile to develop a new parser with a modern architecture, that maybe can also challenge the current integrated RDB parser of Redis and even replace it in the future. So, the below PRS are to replace rdbtools with librdb's tool rdb-cli. sonic-net/sonic-buildimage#19268 sonic-net#1391
…sting by replacing redis-rdb-tool with rdb-cli (sonic-net#1391) Why I did it Fix issue: sonic-net#1387 The latest redis-rdb-tools-0.1.15 doesn't support Redis 7.0. Redis 7.0 was released in 2020 and adopted by SONiC's latest version. So, this issue turned out. https://github.com/sripathikrishnan/redis-rdb-tools I.e., the rdb-tools is far behind the Redis 7.0. The librdb can perfectly fix this issue. Please see quote from https://github.com/redis/librdb. Motivation behind this project There is a genuine need by the Redis community for a versatile RDB file parser that can export data, perform data analysis, or merely extract raw data from RDB and RESTORE it against a live Redis server. However, available parsers have shortcomings in some aspects such as lack of long-term support, lagging far behind the latest Redis release, and usually not being optimized for memory, performance, or high-traffic streaming for production environments. Additionally, most of them are not written in C, which limits the reuse of Redis components and the potential to contribute back to Redis repo. To address these issues, it is worthwhile to develop a new parser with a modern architecture, that maybe can also challenge the current integrated RDB parser of Redis and even replace it in the future. So, the below PRS are to replace rdbtools with librdb's tool rdb-cli. sonic-net/sonic-buildimage#19268 sonic-net#1391
Why I did it
Fix issue: sonic-net/sonic-sairedis#1387
The latest redis-rdb-tools-0.1.15 doesn't support Redis 7.0. Redis 7.0 was released in 2020 and adopted by SONiC's latest version.
So, this issue turned out.
https://github.com/sripathikrishnan/redis-rdb-tools
I.e., the rdb-tools is far behind the Redis 7.0. The librdb can perfectly fix this issue. Please see quote from https://github.com/redis/librdb.
Motivation behind this project
There is a genuine need by the Redis community for a versatile RDB file parser that can export data, perform data analysis, or merely extract raw data from RDB and RESTORE it against a live Redis server. However, available parsers have shortcomings in some aspects such as lack of long-term support, lagging far behind the latest Redis release, and usually not being optimized for memory, performance, or high-traffic streaming for production environments. Additionally, most of them are not written in C, which limits the reuse of Redis components and the potential to contribute back to Redis repo. To address these issues, it is worthwhile to develop a new parser with a modern architecture, that maybe can also challenge the current integrated RDB parser of Redis and even replace it in the future.
So, the below PRS are to replace rdbtools with librdb's tool rdb-cli.
#19268
sonic-net/sonic-sairedis#1391
The above PRs are based on the below PRs.
sonic-net/sonic-sairedis#1288
sonic-net/sonic-sairedis#1298
#16466
sonic-net/sonic-utilities#2972
How I did it
To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
How to verify it
Method 1:
On T2 setup with more than 96K routes, execute CLI command -- generate_dump
No error should be shown
Method 2:
Go into syncd0 container.
admin@ixre-egl-board30:~$ docker exec -it syncd0 bash
Run below command, get saidump's sha1sum value.
root@ixre-egl-board30:/# saidump | sha1sum
1893097ecaeae72383432a54bfe4b285d05b23e5 -
Run below command, get saidump.sh's sha1sum value.
root@ixre-egl-board30:/# saidump.sh | sha1sum
1893097ecaeae72383432a54bfe4b285d05b23e5 -
The two sha1sum values should be the same and no error should be shown.
Method 3:
To run the testbed show_techsupport test case.
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)