Refactor Project Layout to separate components from shared packages using Golang Guidelines

[Psykepro]

The project's current file structure has too many top-level packages, which makes navigation in the project a little bit hard. Especially for newcomers. We want to have a clear understanding from the first glance of the project, which packages are components and which are reusable libraries/utilities from external applications, and which are non-reusable.

Background

At first glance for someone who is looking at the project, it is not obvious which packages are components and which are shared libraries (reusable and non-reusable by external applications). This also makes navigation through the project a little bit hard with so many packages on the top level.

Also, the current file structure leads to duplicated code in some places. For example, we have the exact same waitTick function in aggregator/aggregator.go and sequence/sequencer.go. Also, some utility functions which are placed in the test package are used inside other non-test packages/files. For example, the WaitTxToBeMined from test/operations/wait.go and another one is GetEnv from test/testutils/utils.go.

Proposal

We suggest following the well-known go-standard guidelines for project layout: https://github.com/golang-standards/project-layout.
Hence we would like to introduce new 3 packages:

• components - containing only the components packages as sub-packages
• internal - containing only the internally used libraries/utilities (non-reusable by external applications)
• pkg - containing only shared libraries/utilities (reusable by external applications)

Here is what it would be like after moving the sub-packages to the introduced 3 new top-level packages:

Collapsed top-level packages:

Expanded top-level packages:

Note: Maybe the package sequencer/broadcast can be moved to the new components folder as it looks like it started as a separate component.

Few examples of things that can work better after applying the project layout:

• waitTick function in aggregator/aggregator.go and sequence/sequencer.go - can be moved as an externally reusable sub-package in pkg/datetime/time.go
• WaitTxToBeMined from test/operations/wait.go - can be moved to the internally reusable sub-package in internal/blockchain/transactions.go
• GetEnv from test/testutils/utils.go - can be moved as an externally reusable sub-package in pkg/env/environment.go and to rename the function to Get because we have the env in the package name.
• lowerCaseFirst from jsonrpc/handler.go - can be moved to an externally reusable sub-package pkg/str/string.go

About the pkg sub-packages we can use the same names as the go standard library but then we will have collisions with their names. For example, for the datetime, env and str
packages, respectively it makes sense to be time, os and strings. And if we want to name them like that we need to use aliases with some convention required. But then we need to be sure we follow it. For example, we can have utils suffixes as follows: timeutils, osutils and stringsutils. Or as another option, the suffix can be attached directly to the package names which collide (which will also be suggested properly by the IntelliSense from the IDE and no convention will need to be followed then).

Please provide any suggestions on that if we can make it better or easier.

Suggested by: @kind84 and @Psykepro