Grype doesn't take account OS distributor patch release #1541

sekveaja · 2023-10-05T22:03:32Z

What happened:

Scan a container that has only python3-lxml-4.7.1-150200.3.10.1.x86_64 installed, then, get the following: CVE-2022-2309

lxml                                         4.7.1                                                                      python        CVE-2022-2309        High
lxml                                         4.7.1                                      4.9.1                       python         GHSA-wrxv-2j5q-m38w  Medium

According to SUSE, CVE-2022-2409 issue has been patched with the following version 4.7.1-150200.3.10.1
See SUSE CVE reference: https://www.suse.com/security/cve/CVE-2022-2309.html

As for NVD, it is expected to have fix from version 4.9.1
https://nvd.nist.gov/vuln/detail/CVE-2022-2309

What you expected to happen:

If Grype compares the currently installed (4.7.1-150200.3.10.1 ) version against NVD resolution version, which is 4.9.1+, that will always generate a vulnerability.

Looking for information inside the container:

A) Verify what is installed in the container related to lmxl

> rpm -qa | grep lxml
    python3-lxml-4.7.1-150200.3.10.1.x86_64

B) Query the information of the package

> rpm -qi python3-lxml-4.7.1-150200.3.10.1.x86_64
   Name        : python3-lxml
   Version     : 4.7.1
   Release     : 150200.3.10.1
   Architecture: x86_64
   Install Date: Wed Jul 19 06:14:17 2023
   Group       : Development/Languages/Python
   Size        : 4833352
   License     : BSD-3-Clause AND GPL-2.0-or-later
   Signature   : RSA/SHA256, Mon Aug 22 12:02:17 2022, Key ID 70af9e8139db7c82
   Source RPM  : python-lxml-4.7.1-150200.3.10.1.src.rpm
   Build Date  : Mon Aug 22 12:00:22 2022

C) Query the list of files in the RPM

> rpm -ql python3-lxml-4.7.1-150200.3.10.1.x86_64
   /usr/lib64/python3.6/site-packages/lxml
   /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info
   /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/PKG-INFO
   /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/SOURCES.txt
   /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/dependency_links.txt
   /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/not-zip-safe
   /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/requires.txt
    ....

D) Verify the content in PKG-INFO

      > cat /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/PKG-INFO
      Metadata-Version: 2.1
      Name: lxml
      Version: 4.7.1
      Summary: Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API.
      Home-page: https://lxml.de/
      Author: lxml dev team
      Author-email: [lxml-dev@lxml.de](mailto:lxml-dev@lxml.de)
      Maintainer: lxml dev team
      Maintainer-email: [lxml-dev@lxml.de](mailto:lxml-dev@lxml.de)

E) Package that has installed file /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/PKG-INFO

> rpm -qf /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/PKG-INFO
python3-lxml-4.7.1-150200.3.10.1.x86_64

F) Run syft to verify the packages

$ syft <custom_image>  | grep lxml
     ...
    lxml                                                                         4.7.1                                               python
    python3-lxml                                                          4.7.1-150200.3.10.1                        rpm

Q1: Is CVE reported on lxml 4.7.1 is related to this file: /usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/PKG-INFO?

If the logic is getting info from PKG-INFO, it miss the Release information in the file.

     Metadata-Version: 2.1
      Name: lxml
      Version: 4.7.1
      Summary: Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API.

But this file is really part of the fix RPM from SUSE see E).

Q2: In this situation, how do you plan to fix this?

Other useful information:

When download and scan directly on the rpm file, it doesn’t show any vulnerability with warning:
“Unable to determine the OS distribution. This may result in missing vulnerabilities. You may specify a distro”

$ grype ./python3-lxml-4.7.1-150200.3.10.1.x86_64.rpm
✔ Vulnerability DB                [updated]
✔ Indexed file system                                                                                               /tmp
✔ Cataloged packages              [1 packages]
✔ Scanned for vulnerabilities     [0 vulnerabilities]
   ├── 0 critical, 0 high, 0 medium, 0 low, 0 negligible
   └── 0 fixed
[0000]  WARN unable to access path="/tmp/systemd-private-363d1309eeb04fccbd61c53ec1208496-chronyd.service-6IVyii": open /tm
[0007]  WARN Unable to determine the OS distribution. This may result in missing vulnerabilities. You may specify a distro
No vulnerabilities found
0.69.1

The right argument for SLES OS distribution ( Thanks to @wagoodman )

$ grype --distro sles:15.4 ./python3-lxml-4.7.1-150200.3.10.1.x86_64.rpm
 ✔ Vulnerability DB                [no update available]
 ✔ Indexed file system                                                                                                                   /tmp
 ✔ Cataloged packages              [1 packages]
 ✔ Scanned for vulnerabilities     [0 vulnerability matches]
   ├── by severity: 0 critical, 0 high, 0 medium, 0 low, 0 negligible
   └── by status:   0 fixed, 0 not-fixed, 0 ignored
[0000]  WARN unable to access path="/tmp/systemd-private-363d1309eeb04fccbd61c53ec1208496-chronyd.service-6IVyii": open /tmp/systemd-private-36
No vulnerabilities found

The text was updated successfully, but these errors were encountered:

wagoodman · 2023-10-12T21:20:06Z

I think this is a matter of the documentation not being as clear as it could be on the input for --distro. Using --distro sles:15.4 should appropriately map to the correct vulnerability data:

Does this address your problem?

sekveaja · 2023-10-13T00:22:09Z

@wagoodman Thank you so much for the hint, that solved one issue for the distro reference.

I will update the finding and correct some information on the initial report information.

sekveaja · 2023-10-16T15:11:31Z

There is a simple way to reproduce the problem at the rpm file and at the rpm extracted contents.

At the rpm level

$ grype --distro sles:15.4 ./python3-lxml-4.7.1-150200.3.10.1.x86_64.rpm
  ...
  No vulnerabilities found

==> No vulnerability at the rpm level, as it has all information relate to the fix release see with rpm -qi python3-lxml-4.7.1-150200.3.10.1.x86_64.rpm

Extract the rpm content and scan

i) $ mkdir -p /tmp/Python3-lxml
ii) $ cd /tmp/Python3-lxml
iii) $ rpm2cpio ../python3-lxml-4.7.1-150200.3.10.1.x86_64.rpm | cpio -idmv
iv) $ grype --distro sles:15.4 ./Python3-lxml

NAME  INSTALLED  FIXED-IN  TYPE    VULNERABILITY        SEVERITY
lxml  4.7.1                python  CVE-2022-2309        High
lxml  4.7.1      4.9.1     python  GHSA-wrxv-2j5q-m38w  Medium

And syft provide this output:

$ syft ./Python3-lxml
  ...
  lxml  4.7.1    python

==> 2 files lead to 4.7.1 from the contents of the rpm.
Python3-lxml/usr/lib64/python3.6/site-packages/lxml/init.py
Python3-lxml/usr/lib64/python3.6/site-packages/lxml-4.7.1-py3.6.egg-info/PKG-INFO

From the OS distributor point of view this version and release python3-lxml-4.7.1-150200.3.10.1.x86_64.rpm has already the CVE fixed.

From the tool point view, it isn't, because not all file has release name "-150200.3.10.1"

Maybe Syft doesn't have --distro option, could not know the fix is released with "4.7.1-150200.3.10.1" , therefore, always stuck to the major version 4.71, and lead Grype to issue CVE for 4.7.1.

Output from Syft:

[0000]  INFO could not identify distro     <---------   
[0000]  INFO cataloging a directory

sekveaja · 2023-10-18T19:47:48Z

This time I'm testing with another OS which is Red Hat 9.2.

CVE-2022-2309, according to Red Hat, is fixed with RHSA-2022:8226
https://access.redhat.com/errata/RHSA-2022:8226
With this rpm version: python3-lxml-4.6.5-3.el9.x86_64.rp

Conclusion
Regardless the OS distributor, the pattern is the same for python packages.
The tool doesn't correlate patch from the OS distributor <Version>-<Patch Release>.
In example for:
SUSE python3-lxml-4.7.1-150200.3.10.1.x86_64.rpm
Red Hat python3-lxml-4.6.5-3.el9.x86_64.rpm

At the end, we always get CVE reported to the problematic <version>.
Although, patch has been released by OS distributor in this form <version>-<patch release>

If Grype could check the current installed file is belong to patch rpm <version>-<patch release>, that would help not to report extra CVE, therefore, less false positive.

Below are steps to reproduce the Red Hat rpm.

Testing at the RPM level

$ grype --distro rhel:9.2 ./python3-lxml-4.6.5-3.el9.x86_64.rpm

NAME          INSTALLED      FIXED-IN  TYPE  VULNERABILITY  SEVERITY
python3-lxml  0:4.6.5-3.el9            rpm   CVE-2022-2309  Medium

Extract the RPM content and scan

$ grype --distro rhel:9.2 ./Red_Hat_python3-lxml

NAME  INSTALLED  FIXED-IN  TYPE    VULNERABILITY        SEVERITY
lxml  4.6.5                  python  CVE-2022-2309        High
lxml  4.6.5      4.9.1     python  GHSA-wrxv-2j5q-m38w  Medium

Run Syft on the content directory

$ syft ./Red_python3-lxml

NAME  VERSION  TYPE
lxml  4.6.5    python

The RPM information regarding the patch from the OS

$ rpm -qi python3-lxml-4.6.5-3.el9.x86_64.rpm
warning: python3-lxml-4.6.5-3.el9.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Name        : python3-lxml
Version     : 4.6.5
Release     : 3.el9                          <---- Patch release information
Architecture: x86_64
Install Date: (not installed)
Group       : Unspecified
Size        : 4351883
License     : BSD and MIT and zlib
Signature   : RSA/SHA256, Tue 02 Aug 2022 11:43:16 PM CEST, Key ID 199e2f91fd431d51
Source RPM  : python-lxml-4.6.5-3.el9.src.rpm
Build Date  : Tue 02 Aug 2022 10:44:41 PM CEST
Build Host  : x86-64-01.build.eng.rdu2.redhat.com
Relocations : (not relocatable)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor      : Red Hat, Inc.
URL         : https://github.com/lxml/lxml
Summary     : XML processing library combining libxml2/libxslt with the ElementTree API

Looking for files that has 4.6.5 info

$ find . | grep -r "4.6.5"
usr/lib64/python3.9/site-packages/lxml/__init__.py:__version__ = "4.6.5"
usr/lib64/python3.9/site-packages/lxml/includes/lxml-version.h:#define LXML_VERSION_STRING "4.6.5"
usr/lib64/python3.9/site-packages/lxml-4.6.5-py3.9.egg-info/PKG-INFO:Version: 4.6.5
usr/lib64/python3.9/site-packages/lxml-4.6.5-py3.9.egg-info/PKG-INFO:        4.6.5 (2021-12-12)
usr/lib64/python3.9/site-packages/lxml-4.6.5-py3.9.egg-info/SOURCES.txt:doc/html/changes-4.6.5.html

tgerla · 2023-12-08T16:04:06Z

If Grype could check the current installed file is belong to patch rpm -, that would help not to report extra CVE, therefore, less false positive.

Hi @sekveaja, if I am understanding your report correctly, you are showing that when you scan the RPM itself, you get the correct results (one Medium vulnerability), and when you scan the contents of the manually-extracted RPM, you get an additional "High" vulnerability, which is a false positive, since the vulnerability has been backported to the older version. Right?

I believe this is expected behavior, since as you have shown, when you scan the contents of the RPM but not the RPM itself, there is no information available for Syft or Grype to determine that the files came from an RPM.

Around Grype 0.66 we added logic to filter out files provided by distro packages (#1387) -- so when you scan a container image, we should be doing the right thing and taking into account the distro's backports.

I believe the results from your RHEL 9.2 experiment is expected behavior. If you are using a recent version of Grype and scanning a container image that includes this lxml package as an RPM, you should not see this false positive. Can you make sure I am understanding the issue correctly, and can you tell us what version of Grype you are using to scan the container images?

Thanks!

sekveaja · 2023-12-13T21:17:05Z

Hi @tgerla,

Thank you for the explanation, what has been described in (#1387) is pretty much the same general issue with python-<any_package> that I experienced. I was using grype 0.69.1 on the previous test.

To continue with my Red Hat example which has the same issue with SLES.
I test now with Grype 0.73.4

$ grype version
Application:         grype
Version:             0.73.4

Test directly grype with the same rpm

$ grype ./python3-lxml-4.6.5-3.el9.x86_64.rpm
  
   No vulnerabilities found

Comment: At first glance, with no vulnerability found, it seem to fix the issue. But wait to see test 2).

Test with distro association

     $ grype --distro rhel:9.2 ./python3-lxml-4.6.5-3.el9.x86_64.rpm

     NAME             INSTALLED          FIXED-IN  TYPE  VULNERABILITY  SEVERITY
     python3-lxml  0:4.6.5-3.el9                         rpm       CVE-2022-2309  Medium

Comment: This is a false positive, according to REDHAT adviser https://access.redhat.com/errata/RHSA-2022:8226
This issue is fixed with this rpm version: python3-lxml-4.6.5-3.el9.x86_64.rpm
For this case at the RPM level, you can view the info (rpm -qi ) that you had patch from the OS distributor.

Now extract the content of python3-lxml-4.6.5-3.el9.x86_64.rpm

      mkdir -p /tmp/python3-lxml-rhel; cd python3-lxml-rhel
      rpm2cpio ../python3-lxml-4.6.5-3.el9.x86_64.rpm | cpio -idmv

      $ grype --distro rhel:9.2 /tmp/python3-lxml-rhel

        NAME  INSTALLED  FIXED-IN  TYPE    VULNERABILITY        SEVERITY
        lxml  4.6.5      4.9.1     python  GHSA-wrxv-2j5q-m38w  Medium

Comment: The good thing with this new test, we have one package less
Where before we had as followed:

                      NAME  INSTALLED  FIXED-IN  TYPE    VULNERABILITY        SEVERITY
                       lxml  4.6.5                                python  CVE-2022-2309        High
                       lxml  4.6.5                   4.9.1     python  GHSA-wrxv-2j5q-m38w  Medium

It eliminates "lxml 4.6.5 python CVE-2022-2309" but still have this vulnerability GHSA-wrxv-2j5q-m38w, which is fact related to CVE-2022-2309.
But according the OS distributor, it is already fixed with specific "version-release" i.e. 4.6.5-3.
The logic is still sticking to version number comparison 4.6.5 < 4.9.1 = vulnerable, it doesn't considered OS patch release.
Therefore, that vulnerability is still a false positive.

This is probably the nature of Python package, once it is extracted, you won't see the "-release" information in their directory or files.
I believe for "python-<package_name>", we will always end up to this situation, unless, future implementation can check file which RPM is belonged to and if there is patched information on the RPM.

I hope not to bring too much confusion, just want to contribute to improve Grype.

Thanks!

tgerla · 2024-01-04T22:15:15Z

Hey @sekveaja, thanks for the updates. This helps a lot. Most of the behavior you're seeing is expected--differences between scans of the RPM itself and the directory of unpacked RPM contents is expected. But you are correct that the vulnerability shown in test 2 is indeed a false positive.

We've dug into this pretty deeply and we have found an issue with the way we handle Flatpak-based package. We will be opening another issue to track this in the vunnel project, which is the tool responsible for parsing vulnerability data into a Grype database. Please stay tuned, and thank you very much for the analysis! It has helped us find some subtle issues.

wagoodman · 2024-01-04T22:25:09Z

FYI we've captured the flatpak-related issue here anchore/vunnel#443 , which is a separate class of FP that is mentioned/discovered here but doesn't relate to the description.

wagoodman · 2024-07-24T15:41:44Z

In terms of case 2, I'm able to reproduce with an old database

❯ bin/grype db import ./vulnerability-db_v5_2023-12-01T01%3A29%3A12Z_f341711d2e03116140a1.tar.gz
Vulnerability database imported

❯ GRYPE_DB_AUTO_UPDATE=false GRYPE_DB_MAX_ALLOWED_BUILT_AGE=300000h  ./bin/grype --distro rhel:9.2 python3-lxml-4.6.5-3.el9.x86_64.rpm
 ✔ Indexed file system                                                                                                          /Users/wagoodman/scratch/grype-1541
 ✔ Scanned for vulnerabilities     [1 vulnerability matches]
   ├── by severity: 0 critical, 0 high, 1 medium, 0 low, 0 negligible
   └── by status:   0 fixed, 1 not-fixed, 0 ignored
NAME          INSTALLED      FIXED-IN  TYPE  VULNERABILITY  SEVERITY
python3-lxml  0:4.6.5-3.el9            rpm   CVE-2022-2309  Medium

However with the latest DB and grype version I am not able to reproduce:

❯ grype ./python3-lxml-4.6.5-3.el9.x86_64.rpm --distro rhel:9.2
 ✔ Vulnerability DB                [updated]
 ✔ Indexed file system                                                                                                          /Users/wagoodman/scratch/grype-1541
 ✔ Cataloged contents                                                                              359ed8c9a9213625c46bba58b564829e85667999df9f339cdc85eb1ca916d995
   ├── ✔ Packages                        [1 packages]
   └── ✔ Executables                     [0 executables]
 ✔ Scanned for vulnerabilities     [0 vulnerability matches]
   ├── by severity: 0 critical, 0 high, 0 medium, 0 low, 0 negligible
   └── by status:   0 fixed, 0 not-fixed, 0 ignored
No vulnerabilities found

I think this means that the last open problem described in this issue is resolved 🎉 Please feel free to reopen if that is not the case

sekveaja added the bug Something isn't working label Oct 5, 2023

tgerla self-assigned this Dec 7, 2023

sekveaja mentioned this issue Dec 22, 2023

False Positive: CVE-2023-25577 python3-Werkzeug-1.0.1-150300.3.3.1 #1536

Closed

wagoodman mentioned this issue Jan 4, 2024

Redhat package_name with / do not always reference modules anchore/vunnel#443

Closed

wagoodman closed this as completed Jul 24, 2024

wagoodman added the changelog-ignore Don't include this issue in the release changelog label Jul 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Grype doesn't take account OS distributor patch release #1541

Grype doesn't take account OS distributor patch release #1541

sekveaja commented Oct 5, 2023 •

edited by wagoodman

Loading

wagoodman commented Oct 12, 2023

sekveaja commented Oct 13, 2023 •

edited

Loading

sekveaja commented Oct 16, 2023 •

edited by wagoodman

Loading

sekveaja commented Oct 18, 2023 •

edited by wagoodman

Loading

tgerla commented Dec 8, 2023

sekveaja commented Dec 13, 2023 •

edited by wagoodman

Loading

tgerla commented Jan 4, 2024

wagoodman commented Jan 4, 2024

wagoodman commented Jul 24, 2024

Grype doesn't take account OS distributor patch release #1541

Grype doesn't take account OS distributor patch release #1541

Comments

sekveaja commented Oct 5, 2023 • edited by wagoodman Loading

Looking for information inside the container:

Other useful information:

wagoodman commented Oct 12, 2023

sekveaja commented Oct 13, 2023 • edited Loading

sekveaja commented Oct 16, 2023 • edited by wagoodman Loading

sekveaja commented Oct 18, 2023 • edited by wagoodman Loading

tgerla commented Dec 8, 2023

sekveaja commented Dec 13, 2023 • edited by wagoodman Loading

tgerla commented Jan 4, 2024

wagoodman commented Jan 4, 2024

wagoodman commented Jul 24, 2024

sekveaja commented Oct 5, 2023 •

edited by wagoodman

Loading

sekveaja commented Oct 13, 2023 •

edited

Loading

sekveaja commented Oct 16, 2023 •

edited by wagoodman

Loading

sekveaja commented Oct 18, 2023 •

edited by wagoodman

Loading

sekveaja commented Dec 13, 2023 •

edited by wagoodman

Loading