DAOS-13777 control: Invalid storage information with MD-on-SSD #12485

knard-intel · 2023-06-22T13:19:05Z

Description

With storage query usage, only take into account NVMe devices which are used to store data.

Before requesting gatekeeper:

Two review approvals and any prior change requests have been resolved.
Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
Commit messages follows the guidelines outlined here.
Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

With storage query usage, only take into account NVMe devices which are used to store data. Features: control Required-githooks: true Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@intel.com>

github-actions · 2023-06-22T13:19:25Z

Bug-tracker data:
Ticket title is 'Invalid storage information with MD-on-SSD'
Status is 'In Review'
Labels: 'md_on_ssd,triaged'
https://daosio.atlassian.net/browse/DAOS-13777

daosbuild1

LGTM. No errors found by checkpatch.

kjacque

Seems reasonable, but I'd like to see some unit test coverage that exercises the changes.

daosbuild1 · 2023-06-24T02:47:16Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12485/1/execution/node/1286/log

knard-intel · 2023-06-26T06:42:28Z

Seems reasonable, but I'd like to see some unit test coverage that exercises the changes.

Add unit tests covering the change

tanabarr

LGTM, notwithstanding the comment about a unit test case added to verify the change

…/daos-13777 Required-githooks: true

…/daos-13777

Integrate revivewers comments. Features: control Required-githooks: true Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@intel.com>

daosbuild1

LGTM. No errors found by checkpatch.

kjacque · 2023-06-26T15:50:13Z

src/control/lib/control/pool.go

@@ -1266,6 +1266,11 @@ func processSCMSpaceStats(log logging.Logger, filterRank filterRankFn, scmNamesp
 func processNVMeSpaceStats(log logging.Logger, filterRank filterRankFn, nvmeControllers storage.NvmeControllers, rankNVMeFreeSpace rankFreeSpaceMap) error {
 for _, nvmeController := range nvmeControllers {
 for _, smdDevice := range nvmeController.SmdDevices {
+ if !smdDevice.Roles.IsEmpty() && (smdDevice.Roles.OptionBits&storage.BdevRoleData) == 0 {


Good change for ctl_storage_rpc_test.go, but I was hoping to see a test case for this one as well, showing that we now skip the non-data SSDs.

Add unit tests for processNVMeSpaceStats() function

Thanks for the remark.
The unit test allowed me to fix a potential issue.
Indeed, the device was not skipped: forgot the continue statement.
However, as the usable size was reset to zero in the control plane it was not detected with the functional tests.
With future update of the control plane, this last action could have not been done any more and a bug would arise.

daosbuild1 · 2023-06-27T06:36:22Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12485/2/execution/node/1286/log

…/daos-13777 Required-githooks: true

Integrate revivewers comments: * Add unit tests for processNVMeSpaceStats() function #12485 (comment) Features: control Required-githooks: true Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@intel.com>

daosbuild1

LGTM. No errors found by checkpatch.

daosbuild1 · 2023-06-29T12:27:03Z

Test stage Functional on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12485/3/execution/node/1096/log

daosbuild1

LGTM. No errors found by checkpatch.

…/daos-13777 Test-tags: pr control,-version_number,-test_daos_control_config_basic,-test_daos_agent_config_basic Required-githooks: true

knard-intel · 2023-07-06T08:23:08Z

Resubmit to CI with skipping functional tests related to the JIRA ticket DAOS-13886

daosbuild1

LGTM. No errors found by checkpatch.

daosbuild1 · 2023-07-06T17:53:40Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12485/11/execution/node/1286/log

daosbuild1 · 2023-07-06T18:20:28Z

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12485/11/execution/node/1241/log

daosbuild1

LGTM. No errors found by checkpatch.

daosbuild1 · 2023-07-07T23:35:23Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12485/12/execution/node/1270/log

knard-intel · 2023-07-10T08:05:25Z

PR Build #11 failure:

DaosAggregationBasic:test_basic_aggregation: Failure on too small test timeout => should be a CI issue.
DaosAggregationBasic:test_basic_aggregation: Failure on too small test timeout => should be a CI issue.
AggregationChecksum:test_aggregationchecksum: Failure on too small test timeout => should be a CI issue.
PoolCreateTests:test_create_no_space_loop: CI issue DAOS-13582 under investigation
DaosCoreTest:test_daos_extend_simple: Failure on too small test timeout => should be a CI issue.
DaosCoreTest:test_daos_extend_simple: Same issue as previous. Missing cmocka results of the previous test.

daosbuild1

LGTM. No errors found by checkpatch.

daosbuild1 · 2023-07-10T14:07:52Z

Test stage Unit Test on EL 8 completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-12485/13/testReport/

…/daos-13777 Test-tag: pr control,-version_number,-test_daos_control_config_basic,-test_daos_agent_config_basic Required-githooks: true

daosbuild1

LGTM. No errors found by checkpatch.

daosbuild1 · 2023-07-11T21:14:26Z

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12485/14/execution/node/1241/log

…/daos-13777 Test-tags: pr control,-version_number,-test_daos_control_config_basic,-test_daos_agent_config_basic Required-githooks: true

daosbuild1

LGTM. No errors found by checkpatch.

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master/daos-13777 Required-githooks: true

daosbuild1

LGTM. No errors found by checkpatch.

daosbuild1 · 2023-07-13T19:28:41Z

Test stage Functional Hardware Medium Verbs Provider completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-12485/16/testReport/

knard-intel · 2023-07-14T06:15:40Z

The only test failing is a known CI issue:

DAOS-13914: issue of the last build daos_test.DAOS_Rebuild_Simple

DAOS-13777 control: Invalid storage information with MD-on-SSD

9e25787

With storage query usage, only take into account NVMe devices which are used to store data. Features: control Required-githooks: true Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@intel.com>

daosbuild1 reviewed Jun 22, 2023

View reviewed changes

knard-intel marked this pull request as ready for review June 23, 2023 20:30

knard-intel requested a review from a team as a code owner June 23, 2023 20:30

knard-intel requested review from mjmac, tanabarr and kjacque and removed request for a team June 23, 2023 20:30

kjacque reviewed Jun 24, 2023

View reviewed changes

tanabarr reviewed Jun 26, 2023

View reviewed changes

tanabarr previously approved these changes Jun 26, 2023

View reviewed changes

kanard38 added 3 commits June 26, 2023 14:46

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master…

db4fd3e

…/daos-13777 Required-githooks: true

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master…

bc2f5aa

…/daos-13777

DAOS-13777 control: Invalid storage information with MD-on-SSD

e75b2ef

Integrate revivewers comments. Features: control Required-githooks: true Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@intel.com>

knard-intel dismissed tanabarr’s stale review via e75b2ef June 26, 2023 15:27

knard-intel requested review from kjacque and tanabarr June 26, 2023 15:28

daosbuild1 reviewed Jun 26, 2023

View reviewed changes

kjacque reviewed Jun 26, 2023

View reviewed changes

kanard38 added 2 commits June 29, 2023 08:02

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master…

3b37d17

…/daos-13777 Required-githooks: true

DAOS-13777 control: Invalid storage information with MD-on-SSD

fa65e00

Integrate revivewers comments: * Add unit tests for processNVMeSpaceStats() function #12485 (comment) Features: control Required-githooks: true Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@intel.com>

daosbuild1 reviewed Jun 29, 2023

View reviewed changes

knard-intel requested a review from a team as a code owner June 30, 2023 07:33

knard-intel requested a review from kjacque June 30, 2023 07:34

daosbuild1 reviewed Jun 30, 2023

View reviewed changes

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master…

f069033

…/daos-13777 Test-tags: pr control,-version_number,-test_daos_control_config_basic,-test_daos_agent_config_basic Required-githooks: true

daosbuild1 reviewed Jul 6, 2023

View reviewed changes

knard-intel self-assigned this Jul 6, 2023

knard-intel added the bug label Jul 6, 2023

daosbuild1 reviewed Jul 7, 2023

View reviewed changes

knard-intel requested review from daltonbohning and kjacque July 7, 2023 17:47

tanabarr approved these changes Jul 10, 2023

View reviewed changes

daosbuild1 reviewed Jul 10, 2023

View reviewed changes

kjacque approved these changes Jul 10, 2023

View reviewed changes

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master…

969597e

…/daos-13777 Test-tag: pr control,-version_number,-test_daos_control_config_basic,-test_daos_agent_config_basic Required-githooks: true

daosbuild1 reviewed Jul 11, 2023

View reviewed changes

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master…

070a496

…/daos-13777 Test-tags: pr control,-version_number,-test_daos_control_config_basic,-test_daos_agent_config_basic Required-githooks: true

daosbuild1 reviewed Jul 12, 2023

View reviewed changes

Features: control

b9e66f8

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master/daos-13777 Required-githooks: true

daosbuild1 reviewed Jul 13, 2023

View reviewed changes

mjmac approved these changes Jul 13, 2023

View reviewed changes

knard-intel requested a review from a team July 14, 2023 06:15

knard-intel added the forced-landing The PR has known failures or has intentionally reduced testing, but should still be landed. label Jul 14, 2023

mchaarawi merged commit d502996 into master Jul 17, 2023
15 checks passed

mchaarawi deleted the ckochhof/fix/master/daos-13777 branch July 17, 2023 13:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DAOS-13777 control: Invalid storage information with MD-on-SSD #12485

DAOS-13777 control: Invalid storage information with MD-on-SSD #12485

knard-intel commented Jun 22, 2023

github-actions bot commented Jun 22, 2023 •

edited

Loading

daosbuild1 left a comment

kjacque left a comment

daosbuild1 commented Jun 24, 2023

knard-intel commented Jun 26, 2023 •

edited

Loading

tanabarr left a comment •

edited

Loading

daosbuild1 left a comment

kjacque Jun 26, 2023

knard-intel Jun 29, 2023 •

edited

Loading

knard-intel Jun 29, 2023

daosbuild1 commented Jun 27, 2023

daosbuild1 left a comment

daosbuild1 commented Jun 29, 2023

daosbuild1 left a comment

knard-intel commented Jul 6, 2023

daosbuild1 left a comment

daosbuild1 commented Jul 6, 2023

daosbuild1 commented Jul 6, 2023

daosbuild1 left a comment

daosbuild1 commented Jul 7, 2023

knard-intel commented Jul 10, 2023

daosbuild1 left a comment

daosbuild1 commented Jul 10, 2023

daosbuild1 left a comment

daosbuild1 commented Jul 11, 2023

daosbuild1 left a comment

daosbuild1 left a comment

daosbuild1 commented Jul 13, 2023

knard-intel commented Jul 14, 2023

DAOS-13777 control: Invalid storage information with MD-on-SSD #12485

DAOS-13777 control: Invalid storage information with MD-on-SSD #12485

Conversation

knard-intel commented Jun 22, 2023

Description

Before requesting gatekeeper:

Gatekeeper:

github-actions bot commented Jun 22, 2023 • edited Loading

daosbuild1 left a comment

Choose a reason for hiding this comment

kjacque left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jun 24, 2023

knard-intel commented Jun 26, 2023 • edited Loading

tanabarr left a comment • edited Loading

Choose a reason for hiding this comment

daosbuild1 left a comment

Choose a reason for hiding this comment

kjacque Jun 26, 2023

Choose a reason for hiding this comment

knard-intel Jun 29, 2023 • edited Loading

Choose a reason for hiding this comment

knard-intel Jun 29, 2023

Choose a reason for hiding this comment

daosbuild1 commented Jun 27, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jun 29, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

knard-intel commented Jul 6, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jul 6, 2023

daosbuild1 commented Jul 6, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jul 7, 2023

knard-intel commented Jul 10, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jul 10, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jul 11, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jul 13, 2023

knard-intel commented Jul 14, 2023

github-actions bot commented Jun 22, 2023 •

edited

Loading

knard-intel commented Jun 26, 2023 •

edited

Loading

tanabarr left a comment •

edited

Loading

knard-intel Jun 29, 2023 •

edited

Loading