Fix flake in detecting reorg by nodes

[b-gopalswami]

For some weird reasons, the health check call in nodes doesn't show finality violated error consistently across all the nodes. Due to which this test TestSmokeCCIPReorgAboveFinalityAtDestination/Above_finality_reorg_in_destination_chain flakes often in CI.

Adjusting the expectation to have at least half of the nodes detects violation instead of every node.
CI Failure examples: https://github.com/smartcontractkit/chainlink/actions/runs/11894867899/job/33143516553
https://github.com/smartcontractkit/chainlink/actions/runs/11894232425/job/33141414438?pr=15282

[AnieeG]

should we adjust this to half of the nodes just to make the tests pass? I think it will help if we first try to investigate why rest of the nodes are not able to detect it? Also why half? if we verify f+1 number is able to detect reorg I think it would make more sense.

[b-gopalswami]

should we adjust this to half of the nodes just to make the tests pass? I think it will help if we first try to investigate why rest of the nodes are not able to detect it? Also why half? if we verify f+1 number is able to detect reorg I think it would make more sense.

@AnieeG My understanding on this even if one node detects violation, the system should get halted. So, I was trying to make sure at least half the nodes detects it. If that's not the case and we need f+1 node to detect, then it would make sense to update.

I couldn't reproduce this failure in my local and node logs are not available in CI run. As every nodes are setup similarly in this, wondering why some nodes alone behave different or this might be due to timings?

[AnieeG]

should we adjust this to half of the nodes just to make the tests pass? I think it will help if we first try to investigate why rest of the nodes are not able to detect it? Also why half? if we verify f+1 number is able to detect reorg I think it would make more sense.

@AnieeG My understanding on this even if one node detects violation, the system should get halted. So, I was trying to make sure at least half the nodes detects it. If that's not the case and we need f+1 node to detect, then it would make sense to update.

I couldn't reproduce this failure in my local and node logs are not available in CI run. As every nodes are setup similarly in this, wondering why some nodes alone behave different or this might be due to timings?

@mateusz-sekara is this right assumption?

My understanding on this even if one node detects violation, the system should get halted.

[makramkd]

No, enough nodes need to detect it and avoid participating in OCR. Commit requires 2f+1 observations, so f+1 nodes need to detect it in order to force the entire DON to stop processing messages.

Half the committee is sort of that, e.g for f = 1, N = 4 and half the committee means 2 nodes are seeing finality violations, which is also f+1. For f = 2, N = 7 and ceil(N/2) = 4 which is f+2.