Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(tracing): Fix add_query_source with modules outside of project root #3313

Merged
merged 5 commits into from
Oct 1, 2024

Conversation

rominf
Copy link
Contributor

@rominf rominf commented Jul 19, 2024

Fix: #3312

Previously, when packages added in in_app_include were installed to a location outside of the project root directory, span from those packages were not extended with OTel compatible source code information. Cases include running Python from virtualenv created outside of the project root directory or Python packages installed into the system using package managers. This resulted in an inconsistency: spans from the same project would be different, depending on the deployment method.

The first change moves should_be_included logic into the function to enable unit testing.

In the second change, the logic was slightly changed to avoid these discrepancies and conform to the requirements, described in the PR with a better setting of in-app in stack frames: #1894 (comment).


General Notes

Thank you for contributing to sentry-python!

Please add tests to validate your changes, and lint your code using tox -e linters.

Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.

For maintainers

Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, please carefully check the PR. Then, apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.

@antonpirker
Copy link
Member

Hey @rominf ,
I just glanced over it, but thanks for this contribution and having a great description in issue and PR!
We will look at this next week!

@rominf rominf force-pushed the rominf-tracing-fix-logic-in_app branch from 2d74969 to a6921a5 Compare July 19, 2024 12:16
@antonpirker
Copy link
Member

FYI @rominf ignore the failing AWS tests, they need to be started by us for contributions from outside our organisation.

@rominf
Copy link
Contributor Author

rominf commented Jul 19, 2024

I noticed some mistakes in the first commit message in the reproduction steps. Since I've fixed them in the issue, I'll push updated commits when/if needed, to avoid extra CI runs.

Copy link

codecov bot commented Jul 19, 2024

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
110 4 106 0
View the full list of 3 ❄️ flaky tests
tests.integrations.aws_lambda.test_aws test_init_error[python3.12]

Flake rate in main: 61.11% (Passed 7 times, Failed 11 times)

Stack Traces | 10.7s run time
.../integrations/aws_lambda/test_aws.py:335: in test_init_error
    event = envelope_items[-1]
E   IndexError: list index out of range
tests.integrations.aws_lambda.test_aws test_init_error[python3.9]

Flake rate in main: 61.11% (Passed 7 times, Failed 11 times)

Stack Traces | 11.5s run time
.../integrations/aws_lambda/test_aws.py:335: in test_init_error
    event = envelope_items[-1]
E   IndexError: list index out of range
tests.integrations.aws_lambda.test_aws test_init_error[python3.11]

Flake rate in main: 61.11% (Passed 7 times, Failed 11 times)

Stack Traces | 11.6s run time
.../integrations/aws_lambda/test_aws.py:335: in test_init_error
    event = envelope_items[-1]
E   IndexError: list index out of range

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

@rominf rominf force-pushed the rominf-tracing-fix-logic-in_app branch 3 times, most recently from d8702c8 to 7ff3956 Compare July 20, 2024 07:31
@rominf rominf force-pushed the rominf-tracing-fix-logic-in_app branch from 7ff3956 to 6b68fd1 Compare July 30, 2024 11:16
sentry_sdk/tracing_utils.py Outdated Show resolved Hide resolved
rominf added a commit to rominf/sentry-python that referenced this pull request Aug 20, 2024
Preparation for:
getsentry#3313
Proposed in:
getsentry#3313 (comment)

Note that the `_module_in_list` function returns `False` if `name` is
`None` or `items` are falsy, hence extra check before function call can
be omitted to simplify code.
rominf added a commit to rominf/sentry-python that referenced this pull request Aug 20, 2024
Preparation for:
getsentry#3313
Proposed in:
getsentry#3313 (comment)

Note that the `_module_in_list` function returns `False` if `name` is
`None` or `items` are falsy, hence extra check before function call can
be omitted to simplify code.
szokeasaurusrex added a commit that referenced this pull request Aug 27, 2024
* chore(tracing): Refactor `tracing_utils.py`

Preparation for:
#3313
Proposed in:
#3313 (comment)

Note that the `_module_in_list` function returns `False` if `name` is
`None` or `items` are falsy, hence extra check before function call can
be omitted to simplify code.

* ref: Further simplify `should_be_included` logic

---------

Co-authored-by: Daniel Szoke <daniel.szoke@sentry.io>
@rominf rominf force-pushed the rominf-tracing-fix-logic-in_app branch 3 times, most recently from 5f7042d to 6fa39db Compare August 27, 2024 18:47
@rominf rominf marked this pull request as draft August 27, 2024 18:48
@rominf rominf marked this pull request as ready for review August 27, 2024 18:57
@rominf
Copy link
Contributor Author

rominf commented Aug 27, 2024

Comments have been addressed, and the code has become much simpler than in the first iteration. Thank you, @szokeasaurusrex, for asking me to split my PR into two (refactoring and actual logic change)!

The change is required for unit testing the logic.
…root

Fix: getsentry#3312

Previously, when packages added in `in_app_include` were installed
to a location outside of the project root directory, span from
those packages were not extended with OTel compatible source code
information. Cases include running Python from virtualenv created
outside of the project root directory or Python packages installed into
the system using package managers. This resulted in an inconsistency:
spans from the same project would be different, depending on the
deployment method.

In this change, the logic was slightly changed to avoid these
discrepancies and conform to the requirements, described in the PR with
better setting of in-app in stack frames:
getsentry#1894 (comment).
@rominf rominf force-pushed the rominf-tracing-fix-logic-in_app branch from 6fa39db to c6ce9c7 Compare September 5, 2024 17:59
Copy link
Member

@szokeasaurusrex szokeasaurusrex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @rominf, I am a bit confused by this PR now that you split the PR into two pieces.

Has the logic changed at all in this PR, and if so, then how? Or, is this simply a refactor of the existing logic, plus a new test?

@rominf
Copy link
Contributor Author

rominf commented Sep 10, 2024

Hi @szokeasaurusrex. Yes, the logic has changed. Previously when packages added in in_app_include were installed to a location outside of the project root directory, span from those packages were not extended with OTel compatible source code information. This issue and reproduction steps are described in #3312.

In the first commit I introduce testing suite with one test marked with xfail, as it should pass according to my understanding of the requirements described in the comment of introduction PR, but it does not pass currently. You can check the failure of meeting requirements by doing a checkout of the first commit, running the testing suite, and observing XFAIL instead of XPASS:

$ git checkout -d rominf-tracing-fix-logic-in_app~
$ pytest tests/test_tracing_utils.py
...
tests/test_tracing_utils.py::test_should_be_included[Frame from system-wide installed Django with `django` in `in_app_include`-True] XFAIL  # <--- XFAIL shows that the test with marker `xfail` failed

In the second commit I update the program logic to make it pass and remove xfail marker (as you can see, all tests pass now). I split should_be_included to two variables (should_be_included and should_be_excluded) to model the requirements referenced above.

I believe it is easier to review the logic in the commits by comparing the requirements with the code and checking that all cases are covered, but not by looking at the diff. If you think the logic is not fully covered by tests, please tell me about my oversight – I am open to add yet another test(s) for full coverage.

Copy link
Member

@szokeasaurusrex szokeasaurusrex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks for the explanation, I can see the difference now! Gonna make a few finishing touches, then merge

sentry_sdk/tracing_utils.py Outdated Show resolved Hide resolved
@szokeasaurusrex szokeasaurusrex added the Trigger: tests using secrets PR code is safe; run CI label Sep 23, 2024
arjennienhuis pushed a commit to arjennienhuis/sentry-python that referenced this pull request Sep 30, 2024
* chore(tracing): Refactor `tracing_utils.py`

Preparation for:
getsentry#3313
Proposed in:
getsentry#3313 (comment)

Note that the `_module_in_list` function returns `False` if `name` is
`None` or `items` are falsy, hence extra check before function call can
be omitted to simplify code.

* ref: Further simplify `should_be_included` logic

---------

Co-authored-by: Daniel Szoke <daniel.szoke@sentry.io>
@sentrivana sentrivana enabled auto-merge (squash) October 1, 2024 09:09
@github-actions github-actions bot removed the Trigger: tests using secrets PR code is safe; run CI label Oct 1, 2024
@sentrivana sentrivana added the Trigger: tests using secrets PR code is safe; run CI label Oct 1, 2024
@sentrivana sentrivana merged commit 4636afc into getsentry:master Oct 1, 2024
138 of 140 checks passed
@rominf rominf deleted the rominf-tracing-fix-logic-in_app branch October 1, 2024 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Trigger: tests using secrets PR code is safe; run CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

in_app_include in add_query_source works differently, depending on the package installation method
4 participants