Trivy vuln scan took so long in CI pipeline.

[davidsuryaputra]

Question

I ran vuln scans against repository in my CI pipeline. However, the scan took so long.

I'm not sure what was happening. I attach the debug file trivy-debug-vuln.txt.

I just want to know why it took so long and what need to be done from my side. It run almost 30 minutes, and still continue.

Thanks

Target

Git Repository

Scanner

Vulnerability

Output Format

JSON

Mode

Standalone

Operating System

linux/amd64

Version

v0.55.0

[DmitriyLewen]

Hello @davidsuryaputra

IIUC you are scanning a Java project.

For pom.xml files - Trivy checks remote repositories/maven central for each dependency (child/parent/etc).
It looks like your pom.xml file contains a lot of dependencies.
Getting each dependency takes time...

If you want to avoid this case - you can use a cache for your piplene with a .m2 directory with all required dependencies.
In this case Trivy will not check remote repositories and will take all dependencies from the local directory.

[mgortzak]

I had the exact same problem. For us it was a misconfigured Maven cache in the Bitbucket pipeline.

But the slowness didn't occur before this. It seems to me as if this problem occured when we started using version 0.55.2. It seemed as if all of a sudden Trivy does something else when scanning each dependency.

[davidsuryaputra]

Hello @davidsuryaputra

IIUC you are scanning a Java project.

For pom.xml files - Trivy checks remote repositories/maven central for each dependency (child/parent/etc). It looks like your pom.xml file contains a lot of dependencies. Getting each dependency takes time...

If you want to avoid this case - you can use a cache for your piplene with a .m2 directory with all required dependencies. In this case Trivy will not check remote repositories and will take all dependencies from the local directory.

Hi @DmitriyLewen

In my case, it is not Java project. it is just repository with few vulnerable apps project like govwa, go-dvwa and terragoat. If the cause is pom.xml then I suspect pom.xml in terragoat project is the source of problem

https://github.com/bridgecrewio/terragoat/blob/master/packages/sub/pom.xml
https://github.com/bridgecrewio/terragoat/blob/master/packages/pom.xml

However, it seems the dependency is not that much. is that normal behaviour in my case? or I could do something to make the scan faster.

Thanks,
David

[DmitriyLewen]

Hello @davidsuryaputra
Trivy checks full tree of dependencies (all child dependencies and their parents).
If local cache doesn't have required pom file - Trivy send request to Maven central.

oh... sorry, I just saw that you're using version v0.55.0. In that version we added test scope scanning.
But we found bug with that and reverted these changes.
Can you check v0.56.1?

[DmitriyLewen]

Hello @mgortzak

problem occured when we started using version 0.55.2

What is version you used before?

We didn't add serious changes in v0.55.2.

But we previously added logic to check release repositories from pom files (#6171)
And updated this logic for snapshot repositories later (#6950).
This can slow down scanning if repositories from pom files don't contain the required artifact.

But all these changes were in the first half of the year.

[mgortzak]

Hi @DmitriyLewen, thank you for your response.

I can't reproduce thus problem anymore, we are running Trivy 0.56.1 now.

This problem did force us to look at how our Bitbucket pipelines worked. We had separate Maven caches; for mvn install, for mvn deploy, etc. The separate Maven cache in the Trivy step didn't have all dependencies need when running trivy. So it started downloading everything. That was a misconfiguration on our own, which we fixed by using a single Maven cache for the entire pipeline. I think this problem was compouned by Trivy scanning more dependencies.

Unfortunately I do not which Trivy version was being used in the pipeline before we started seeing problems.

[DmitriyLewen]

Got it.

Anyway, it's cool that you fixed your pipelines.
If you find information about a previous version of Trivy, let me know and we'll try to figure out the cause of the problem.