Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #12884] Fix Flaky Behavior in NewMetadataSelector Tests #12885

Merged
merged 1 commit into from
Nov 22, 2024

Conversation

DaniellaHubble
Copy link
Contributor

In response to ISSUE #12884

What is the purpose of the change

The purpose is to fix a flakiness issue in the com.alibaba.nacos.client.naming.selector.NamingSelectorFactoryTest.testNewMetadataSelectorand testNewMetadataSelector2 tests, caused by the non-deterministic iteration order of HashMap, which led to inconsistent test outcomes when metadata key-value pairs were accessed.

You can see this issue because testNewMetadataSelector and testNewMetadataSelector2 passed when I ran them normally using the command: mvn -pl client test -Dtest=com.alibaba.nacos.client.naming.selector.NamingSelectorFactoryTest#testNewMetadataSelector but were failing when ran under NonDex, a testing tool that identifies flaky tests caused by non-deterministic behaviors. To run with NonDex, I used the command: mvn -pl client edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=com.alibaba.nacos.client.naming.selector.NamingSelectorFactoryTest#testNewMetadataSelector

Brief changelog

  • Replaced HashMap with LinkedHashMap in testNewMetadataSelector to ensure deterministic iteration order for metadata matching.

  • Explicitly set LinkedHashMap for instance metadata to preserve insertion order during matching.

Verifying this change

To verify this change, I ran the following to execute the tests with NonDex for 10 iterations: mvn -pl client edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=com.alibaba.nacos.client.naming.selector.NamingSelectorFactoryTest#testNewMetadataSelector -DnondexRuns=10
and for the second test I ran:
mvn -pl client edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=com.alibaba.nacos.client.naming.selector.NamingSelectorFactoryTest#testNewMetadataSelector2 -DnondexRuns=10
These commands ensured each test ran 10 times under NonDex. Both testNewMetadataSelector and testNewMetadataSelector2 passed consistently, confirming the absence of flakiness.

Follow this checklist to help us incorporate your contribution quickly and easily:

  • Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
  • Run mvn -B clean package apache-rat:check findbugs:findbugs -Dmaven.test.skip=true to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test to make sure integration-test pass.

@CLAassistant
Copy link

CLAassistant commented Nov 21, 2024

CLA assistant check
All committers have signed the CLA.

Copy link

Thanks for your this PR. 🙏
Please check again for your PR changes whether contains any usage/api/configuration change such as Add new API , Add new configuration, Change default value of configuration.
If so, please add or update documents(markdown type) in docs/next/ for repository nacos-group/nacos-group.github.io


感谢您提交的PR。 🙏
请再次查看您的PR内容,确认是否包含任何使用方式/API/配置参数的变更,如:新增API新增配置参数修改默认配置等操作。
如果是,请确保在提交之前,在仓库nacos-group/nacos-group.github.io中的docs/next/目录下添加或更新文档(markdown格式)。

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.26%. Comparing base (c5e5a82) to head (34ecdda).

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##             develop   #12885      +/-   ##
=============================================
- Coverage      72.27%   72.26%   -0.02%     
  Complexity      9827     9827              
=============================================
  Files           1287     1287              
  Lines          41445    41445              
  Branches        4373     4373              
=============================================
- Hits           29956    29949       -7     
- Misses          9387     9395       +8     
+ Partials        2102     2101       -1     

see 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5e5a82...34ecdda. Read the comment docs.

---- 🚨 Try these New Features:

@wuyfee
Copy link

wuyfee commented Nov 22, 2024

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KomachiSion KomachiSion merged commit 2a0dafa into alibaba:develop Nov 22, 2024
8 checks passed
@KomachiSion KomachiSion added kind/bug Category issues or prs related to bug. area/Test labels Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/Test kind/bug Category issues or prs related to bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants