loader takes longer than 5 mins

[paleboot]

Welcome

• Yes, I'm using a binary release within 2 latest major releases. Only such installations are supported.
• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've read the typecheck section of the FAQ (https://golangci-lint.run/usage/faq/#why-do-you-have-typecheck-errors).
• Yes, I've tried with the standalone linter if available (e.g., gocritic, go vet, etc.). (https://golangci-lint.run/usage/linters/)

Description of the problem

Hello. I'm having troubles using the binary installed in a GitHub Actions runner (not golangci-lint-action) . The gist of it is I have a wrapper that would spawn go routines running the installed binary (exec.Command).

I've tested it on CI, spawning only 2 go routines, but I keep getting timeout, even when I set it to 10 mins. Too long considering I only have 7 linters enabled. Running locally works fine, tested with 3 go routines.

Looking at the debug log it seems to stuck at this process, though I have no idea what it's actually doing or what impacts it (maybe resource?):

level=debug msg="[loader] 5m0.270078796s for GOROOT=/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.5.linux-amd64 GOPATH= GO111MODULE= GOPROXY= PWD=/runner/_work/mydir/mypkg go list -mod=readonly -e -json=Name,ImportPath,Error,Dir,GoFiles,IgnoredGoFiles,IgnoredOtherFiles,CFiles,CgoFiles,CXXFiles,MFiles,HFiles,FFiles,SFiles,SwigFiles,SwigCXXFiles,SysoFiles,CompiledGoFiles,Export,DepOnly,Imports,ImportMap -compiled=true -test=false -export=true -deps=true -find=false -pgo=off -mod=readonly -- ./..."

Thanks in advance!

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version v1.55.2 built with go1.21.1 from (unknown, mod sum: "h1:yllEIsSJ7MtlDBwDJ9IMBkyEUz2fYE0b5B8IUgO1oP8=") on (unknown)

Configuration

linters:
  enable:
    - gocritic
    - gosec
  disable:
    - errcheck
    - interfacer
linters-settings:
  goconst:
    min-occurrences: 5
  govet:
    # report about shadowed variables
    check-shadowing: false
    disable:
      - asmdecl
      - cgocall
      - stdmethods
      - unsafeptr
  goimports:
    # put imports beginning with prefix after 3rd-party packages;
    # it's a comma-separated list of prefixes
    local-prefixes: mydir
  maligned:
    # print struct with more effective memory layout or not, false by default
    suggest-new: true
  misspell:
    # Correct spellings using locale preferences for US or UK.
    # Default is to use a neutral variety of English.
    # Setting locale to US will correct the British spelling of 'colour' to 'color'.
    locale: US
    ignore-words:
      - cancelled
  lll:
    # max line length, lines longer will be reported. Default is 120.
    # '\t' is counted as 1 character by default, and can be changed with the tab-width option
    line-length: 120
    # tab width in spaces. Default to 1.
    tab-width: 1
  revive:
    ignore-generated-header: false
    severity: "warning"
    enable-all-rules: true
    confidence: 0.0
    error-code: 2
    warning-code: 1
    rules:
      - name: add-constant
        disabled: true
      - name: argument-limit
        disabled: true
      - name: banned-characters
        disabled: true
      - name: bare-return
        disabled: true
      - name: blank-imports
        disabled: true
      - name: cognitive-complexity
        disabled: true
      - name: comment-spacings
        disabled: true
      - name: confusing-naming
        disabled: true
      - name: confusing-results
        disabled: true
      - name: cyclomatic
        disabled: true
      - name: datarace
        disabled: true
      - name: deep-exit
        disabled: true
      - name: empty-block
        disabled: true
      - name: empty-lines
        disabled: true
      - name: exported
        disabled: true
      - name: file-header
        disabled: true
      - name: flag-parameter
        disabled: true
      - name: function-result-limit
        disabled: true
      - name: function-length
        disabled: true
      - name: get-return
        disabled: true
      - name: if-return
        disabled: true
      - name: import-shadowing
        disabled: true
      - name: line-length-limit
        disabled: true
      - name: max-public-structs
        disabled: true
      - name: modifies-value-receiver
        disabled: true
      - name: nested-structs
        disabled: true
      - name: optimize-operands-order
        disabled: true
      - name: string-format
        disabled: true
      - name: time-equal
        disabled: true
      - name: var-naming
        disabled: true
      - name: var-declaration
        disabled: true
      - name: unexported-naming
        disabled: true
      - name: unexported-return
        disabled: true
      - name: unhandled-error
        disabled: true
      - name: unnecessary-stmt
        disabled: true
      - name: unused-parameter
        disabled: true
      - name: unused-receiver
        disabled: true
      - name: useless-break
        disabled: true
  unused:
    # treat code as a program (not a library) and report unused exported identifiers; default is false.
    # XXX: if you enable this setting, unused will report a lot of false-positives in text editors:
    # if it's called for subdir of a project it can't find funcs usages. All text editor integrations
    # with golangci-lint call it on a directory with the changed file.
    check-exported: false
  unparam:
    # call graph construction algorithm (cha, rta). In general, use cha for libraries,
    # and rta for programs with main packages. Default is cha.
    algo: cha

    # Inspect exported functions, default is false. Set to true if no external program/library imports your code.
    # XXX: if you enable this setting, unparam will report a lot of false-positives in text editors:
    # if it's called for subdir of a project it can't find external interfaces. All text editor integrations
    # with golangci-lint call it on a directory with the changed file.
    check-exported: false
  gci:
    local-prefixes: mydir
    sections:
      - standard # Captures all standard packages if they do not match another section.
      - default # Contains all imports that could not be matched to another section type.
      - prefix(mydir/) # Groups all imports with the specified Prefix.
  gocritic:
    enabled-checks:
      - appendCombine
      - equalFold
      - indexAlloc
      - rangeExprCopy
      - boolExprSimplify
      - builtinShadow
      - commentedOutCode
      - commentedOutImport
      - docStub
      - emptyFallthrough
      - hexLiteral
      - initClause
      - methodExprCall
      - nilValReturn
      - octalLiteral
      - stringXbytes
      - typeAssertChain
      - typeUnparen
      - unnecessaryBlock
      - weakCond
  gosec:
    excludes:
      - G102
      - G103
      - G104
      - G106
      - G108
      - G110
      - G111
      - G112
      - G113
      - G114
      - G203
      - G204
      - G301
      - G303
      - G304
      - G305
      - G306
      - G402
      - G403
      - G501
      - G502
      - G503
      - G504
      - G505
      - G601
run:
  timeout: 5m
  tests: false
  skip-dirs:
    - vendors
  modules-download-mode: readonly
  allow-parallel-runners: true
  print-resources-usage: true
  go: "1.21.0"

Go environment

$ go version && go env
go version go1.21.5 linux/amd64
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/runner/.cache/go-build'
GOENV='/home/runner/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/runner/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/runner/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.5.linux-amd64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.5.linux-amd64/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.5'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/runner/_work/mydir/go.mod'
GOWORK='/runner/_work/mydir/go.work'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build4055659504=/tmp/go-build -gno-record-gcc-switches'

Verbose output of running

$ golangci-lint cache clean
$ golangci-lint run -v
level=info msg="[config_reader] Used config file ../../../../.config/golangci.yaml"
level=info msg="[lintersdb] Active 7 linters: [gocritic gosec gosimple govet ineffassign staticcheck unused]"
level=debug msg="[loader] Built loader args are [./...]"
level=debug msg="[loader] 4.569436ms for GOROOT=/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.5.linux-amd64 GOPATH= GO111MODULE=off GOPROXY= PWD=/runner/_work/mydir/mypkg go list -e -f {{context.ReleaseTags}} -- unsafe"
level=debug msg="[loader] 8.775452ms for GOROOT=/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.5.linux-amd64 GOPATH= GO111MODULE= GOPROXY= PWD=/runner/_work/mydir/mypkg go list -mod=readonly -f \"{{context.GOARCH}} {{context.Compiler}}\" -- unsafe"
level=info msg="Memory: 2873 samples, avg is 27.1MB, max is 27.1MB"
level=info msg="Execution took 4m59.919971025s"
level=debug msg="[loader] 5m0.270078796s for GOROOT=/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.5.linux-amd64 GOPATH= GO111MODULE= GOPROXY= PWD=/runner/_work/mydir/mypkg go list -mod=readonly -e -json=Name,ImportPath,Error,Dir,GoFiles,IgnoredGoFiles,IgnoredOtherFiles,CFiles,CgoFiles,CXXFiles,MFiles,HFiles,FFiles,SFiles,SwigFiles,SwigCXXFiles,SysoFiles,CompiledGoFiles,Export,DepOnly,Imports,ImportMap -compiled=true -test=false -export=true -deps=true -find=false -pgo=off -mod=readonly -- ./..."
level=info msg="[loader] Go packages loading at mode 575 (imports|name|compiled_files|deps|exports_file|types_sizes|files) took 5m0.275119715s"
level=error msg="Running error: context loading failed: failed to load packages: timed out to load packages: context deadline exceeded"
level=error msg="Timeout exceeded: try increasing it by passing --timeout option"

A minimal reproducible example or link to a public repository

// add your code here

Validation

• Yes, I've included all information above (version, config, etc.).

[boring-cyborg[bot]]

Hey, thank you for opening your first Issue ! 🙂 If you would like to contribute we have a guide for contributors.

[paleboot]

Hi folks, I'm reopening this discussion, curious to know what variables affects the speed of loader. In the original post, the issue was likely to resource starvation i.e. too many goroutines running.

While we had that fixed, our currently running pipeline (only 1 instance of golangci-lint), have sporadic slow load up to 3mins, which ultimately ends with timeout errors albeit not during "loading" but during the entirety of the run.

[edocevol]

Why just give an optional configuration for timeout, but never explain why the timeout occurs? And it looks more like it happened at random