Add GoSec action and fix issues #505

MatheusFranco99 · 2024-09-23T11:58:35Z

Overview

This PR adds the GoSec GitHub action and fixes issues raised by GoSec.

Note that there is still one issue raised by GoSec. This is already solved by #503 . To avoid double work and review efforts, we didn't add it to this PR.

GalRogozinski

Hey,
jsons changed only because of #503?
It is best to have all changes that change spec jsons in seperate PR...

GalRogozinski · 2024-09-24T16:23:31Z

types/beacon_types.go

+	// Sanitize slot duration
+	if slotDurationInSeconds <= 0 {
+		return spec.Slot(math.MaxUint64)
+	}


I am wondering whether we need all of this in spec, especially since Slot Duration should be a constant...
GoSec yells if it isn't there?

I agree. But, yes, GoSec yells (and with reason tbh 😅)

GalRogozinski · 2024-09-24T16:27:37Z

types/beacon_types.go

+	// Get slot duration
+	slotDurationInSeconds := int64(n.SlotDurationSec().Seconds())
+	if slotDurationInSeconds < 0 {
+		return int64(math.MaxInt64)
+	}
+
+	// Get delta to add to genesis time
+	d := uint64(slot) * uint64(slotDurationInSeconds)
+
+	// Get genesis time
+	minGenesisTime := n.MinGenesisTime()
+
+	// Sanitize variables
+	if minGenesisTime > uint64(math.MaxInt64) || d > uint64(math.MaxInt64) {
+		return int64(math.MaxInt64)
+	}
+
+	return int64(minGenesisTime) + int64(d)


converting from uint64 to int64 is considered safe by gosec?
A big number can turn negative I think (no that it will happen but you know)

Notice that, before the conversion, sanity checks ensure it'll be converted properly. GoSec sees this. E.g., if I remove the checks, it will raise the error.

I think... that if the sum of int64(minGenesisTime) + int64(d) will be larger than math.MaxInt64 there will be an overflow... weird that go-sec didn't shout about that...

Anyhow, using some safe arithmetics here is an overkill since we both know that the above scenario will never happen and we don't want to have too many impl details in spec...
So if they shout I will try to talk to them about this

GalRogozinski · 2024-09-24T16:43:56Z

types/testingutils/share_map.go

@@ -17,20 +17,31 @@ func ValidatorIndexList(limit int) []int {
 	return ret
 }

-func KeySetMapForValidatorIndexList(valIndexes []int) map[phase0.ValidatorIndex]*TestKeySet {
+func KeySetMapForValidatorIndexList(valIndexes []phase0.ValidatorIndex) map[phase0.ValidatorIndex]*TestKeySet {


GalRogozinski · 2024-09-24T16:46:47Z

types/testingutils/share_map.go

+	if limit <= 0 {
+		return map[phase0.ValidatorIndex]*TestKeySet{}
+	}
+	validators := make([]phase0.ValidatorIndex, limit)
+	for i := 0; i < limit; i++ {
+		validatorIndex := (i + 1)
+		if validatorIndex < 0 {
+			panic("Invalid validator index")
+		}
+		validators[i] = phase0.ValidatorIndex(validatorIndex)
+	}
+	return KeySetMapForValidatorIndexList(validators)


why not change ValidatorIndexList?

It would incur changes through many tests. I can definitely do it here if you want. But, since it's not the goal of this PR, I decided to leave this for a future PR.

GalRogozinski · 2024-09-24T16:57:05Z

Hmm technically it will be best to simply use correct types in the first place that will require less casting?
It is just that I think this PR made spec a bit less readable... nevertheless I think we should merge the changes and maybe later fix types... since this will put a strain on the engineering team maybe?

MatheusFranco99 · 2024-09-24T22:34:02Z

@GalRogozinski

jsons changed only because of #503?
It is best to have all changes that change spec jsons in seperate PR...

I believe so. Merging 503 first would be great. Since I wanted this PR to pass tests, I had to also change here. If 503 is merged, these changes will disappear.

MatheusFranco99 · 2024-09-24T22:42:05Z

mm technically it will be best to simply use correct types in the first place that will require less casting?

I definitely agree. Some return values should clearly be uint, but they are int and keep requiring casting due to others correctly defined uint types. I decided not to change the functions' outputs because I believe this would incur several other changes on the ssv repo. If this is wanted, and they have time to align with it, I can add it to this PR. But, personally, I agree that we should merge this and later fix these types when they will have time to align.

MatheusFranco99 · 2024-09-25T07:48:49Z

@GalRogozinski Removed the change on GetCommitteeID as you asked for

alan-ssvlabs

We add quite a few (maybe practically meaningless) check for whether ints are negative. Would the better fix be updating them to uints?

types/testingutils/beacon_node.go

GalRogozinski · 2024-09-26T11:06:27Z

@alan-ssvlabs
see #505 (comment)

* remove domaintype from committeeMember * remove domaintype from share * add config for ssv containing domainType * add custom unmarshal for baserunner to avoid error in unmarshalling config * move domainType to Network * add testing domain in NewTestingNetwork function * Spec - filter committee duties per slot validator duties (#467) * Filter shares for slot `CommitteeRunner` based on validators that have duty for that slot. * Filter duty and create share map according to owned validators * Add test: start duty with no shares for duty's validators * Add test: happy flow for committee with fraction of duty's validators * Generate JSON tests * Apply suggestions --------- Co-authored-by: MatheusFranco99 <48058141+MatheusFranco99@users.noreply.github.com> * Spec - check att and sync duties exist before submitting (#468) * Meta - Update to go1.22 (#474) * Update go1.20 to go1.22 * Update go.sum with mod tidy * Meta - Update dependencies (#483) * Update dependencies * Fix lint issue * Generate JSON tests to trigger actions * Update fastssz * Generate JSON tests and align ssz error * Revert go-eth2-client version change * Revert fastssz upgrade * Generate SSZ and JSON tests * Meta - Fix static analysis issues (#480) * Solve potential file inclusion via variable * Fix file permission (0644 to 0600) * Add nosec comment for PRNG (pseudo-random number generator) used for testing * Fix lint issue on nil check in []byte type * Update permission from 0444 to 0600 * Update 0444 to 0400 * Meta - Drop unnecessary nolint comments (#477) * Remove nolint comment and export timeout variables * Drop unnecessary nolint * Add comment * Fix lint issue * Spec - Share length validation (#478) * Add share length validation in runner construction * Align to error handling in runners constructions * Add validation to committee runner * Add runner construction tests * Refactor runner construction in testingutil to deal with creation errors * Generate JSON tests * Fix lint issue * Fix comments * Meta - Drop redundant error (#475) * Spec - Drop redundant validation for decided messages (#476) * Remove redundant validation * Align error string * Spec - Sort signers in decided message (#484) * Sort signers in decided message * Add test for sorted signers in decided msg * Generate JSON tests * Fix lint issue * Spec - Stop processing after decided (#487) * Stop processing consensus messages after instance is decided * Align error in qbft tests * Align errors in ssv tests * Generate JSON tests * Fix lint issue * Spec - Drop leftover error check (#469) * Remove leftover err check * Align argument variable name to type * Spec - Secure key storage (#481) * Implement secure key storage using PKCS8 * Fix lint issue * Switch back PKCS8 to PKCS1 * Meta - Remove residual DKG (#502) * Remove DKG signature type * Remove DKG msg type * Remove DKGOperators field from TestKeySet * Remove unused ecdsaKeys field from TestingKeyStorage * Remove unused "ecdsaSKFromHex" function * Generate JSON tests * Spec - Add GoSec action and fix issues (#505) * Add github action and makefile command * Fix issues in round robin proposer function * Fix bad PutUint32 in GetCommitteeID * Fix issue with HasQuorum and HasPartialQuorum * Add role sanitization in GetRoleType and NewMessageType * Add sanitization to BeaconNetwork methods * Add sanitization in testingutils * Add sanitization to height usage in test files * Fix uint64 conversion in runner/postconsensus/valid_msg test * Sanitize ValidatorIndex conversion * Update action name * Fix tests to use valid RunnerRoles * Generate SSZ * Generate JSON tests * Revert the change on GetCommitteeID * Add nosec G115 to GetCommitteeID * revert the merge because it was merged with origin main by accident --------- Co-authored-by: rehs0y <lyosher@gmail.com> Co-authored-by: MatheusFranco99 <48058141+MatheusFranco99@users.noreply.github.com>

MatheusFranco99 added 10 commits September 23, 2024 12:38

Add github action and makefile command

7da5fac

Fix issues in round robin proposer function

bbd4c14

Fix bad PutUint32 in GetCommitteeID

f949546

Fix issue with HasQuorum and HasPartialQuorum

b2f3430

Add role sanitization in GetRoleType and NewMessageType

9a7f7a0

Add sanitization to BeaconNetwork methods

4c38c33

Add sanitization in testingutils

404f4c0

Add sanitization to height usage in test files

6e412a2

Fix uint64 conversion in runner/postconsensus/valid_msg test

eb52b63

Sanitize ValidatorIndex conversion

4020fbb

MatheusFranco99 requested review from GalRogozinski and alan-ssvlabs September 23, 2024 11:58

MatheusFranco99 self-assigned this Sep 23, 2024

MatheusFranco99 added 4 commits September 23, 2024 13:00

Update action name

33e47a9

Fix tests to use valid RunnerRoles

0ad1208

Generate SSZ

0070719

Generate JSON tests

8bfd270

GalRogozinski reviewed Sep 24, 2024

View reviewed changes

MatheusFranco99 added the spec label Sep 25, 2024

Revert the change on GetCommitteeID

2b008a9

alan-ssvlabs reviewed Sep 25, 2024

View reviewed changes

types/testingutils/beacon_node.go Show resolved Hide resolved

alan-ssvlabs approved these changes Sep 25, 2024

View reviewed changes

Add nosec G115 to GetCommitteeID

54600b9

MatusKysel approved these changes Sep 26, 2024

View reviewed changes

GalRogozinski approved these changes Sep 26, 2024

View reviewed changes

GalRogozinski merged commit 93ad50e into ssvlabs:main Sep 26, 2024
3 checks passed

GalRogozinski deleted the gosec-action branch September 26, 2024 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add GoSec action and fix issues #505

Add GoSec action and fix issues #505

MatheusFranco99 commented Sep 23, 2024 •

edited

Loading

GalRogozinski left a comment

GalRogozinski Sep 24, 2024

MatheusFranco99 Sep 24, 2024

GalRogozinski Sep 24, 2024

MatheusFranco99 Sep 24, 2024 •

edited

Loading

GalRogozinski Sep 26, 2024

GalRogozinski Sep 24, 2024

GalRogozinski Sep 24, 2024

MatheusFranco99 Sep 24, 2024

GalRogozinski commented Sep 24, 2024

MatheusFranco99 commented Sep 24, 2024

MatheusFranco99 commented Sep 24, 2024

MatheusFranco99 commented Sep 25, 2024

alan-ssvlabs left a comment

GalRogozinski commented Sep 26, 2024

Add GoSec action and fix issues #505

Add GoSec action and fix issues #505

Conversation

MatheusFranco99 commented Sep 23, 2024 • edited Loading

Overview

GalRogozinski left a comment

Choose a reason for hiding this comment

GalRogozinski Sep 24, 2024

Choose a reason for hiding this comment

MatheusFranco99 Sep 24, 2024

Choose a reason for hiding this comment

GalRogozinski Sep 24, 2024

Choose a reason for hiding this comment

MatheusFranco99 Sep 24, 2024 • edited Loading

Choose a reason for hiding this comment

GalRogozinski Sep 26, 2024

Choose a reason for hiding this comment

GalRogozinski Sep 24, 2024

Choose a reason for hiding this comment

GalRogozinski Sep 24, 2024

Choose a reason for hiding this comment

MatheusFranco99 Sep 24, 2024

Choose a reason for hiding this comment

GalRogozinski commented Sep 24, 2024

MatheusFranco99 commented Sep 24, 2024

MatheusFranco99 commented Sep 24, 2024

MatheusFranco99 commented Sep 25, 2024

alan-ssvlabs left a comment

Choose a reason for hiding this comment

GalRogozinski commented Sep 26, 2024

MatheusFranco99 commented Sep 23, 2024 •

edited

Loading

MatheusFranco99 Sep 24, 2024 •

edited

Loading