Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

improve baseline matching by using a diff algorithm and comparing the number of lines the diagnostic covers #708

Merged
merged 4 commits into from
Sep 29, 2024
Merged

improve baseline matching by using a diff algorithm and comparing the number of lines the diagnostic covers #708

merged 4 commits into from
Sep 29, 2024

Conversation

DetachHead
Copy link
Owner

No description provided.

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@DetachHead DetachHead changed the title attempt to improve baseline matching improve baseline matching by using a diff algorithm and comparing the line count of the errors Sep 29, 2024
@DetachHead DetachHead marked this pull request as ready for review September 29, 2024 03:21
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@DetachHead DetachHead force-pushed the improve-baseline-matching branch 2 times, most recently from 81a7cda to 3db8dd8 Compare September 29, 2024 04:10
@DetachHead
revert convertDiagnostics back to its cringe any version because …
9d85357 
…somethign somewhere is calling it with the wrong type
@DetachHead DetachHead force-pushed the improve-baseline-matching branch from 3db8dd8 to 9d85357 Compare September 29, 2024 04:27
@DetachHead DetachHead force-pushed the improve-baseline-matching branch from 7aa3dd6 to da794cd Compare September 29, 2024 05:17
@DetachHead DetachHead changed the title improve baseline matching by using a diff algorithm and comparing the line count of the errors improve baseline matching by using a diff algorithm and comparing the number of lines the diagnostic covers Sep 29, 2024
@DetachHead
add unit test for baselined hints and remove TODO for more tests sinc…
a7014f9 
…e the bulk of the matching logic is handled by the diff package
@github-actions GitHub Actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

sympy (https://github.com/sympy/sympy)
-     Type of "sqrtdenest" is "(expr: Unknown, max_iter: int = 3) -> (Unknown | Mul | Pow | Order | Expr | Add | None)" (reportUnknownVariableType)
+     Type of "sqrtdenest" is "(expr: Unknown, max_iter: int = 3) -> Unknown" (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/simplify/tests/test_sqrtdenest.py:123:5 - error: Type of "z1" is unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/simplify/tests/test_sqrtdenest.py:123:5 - error: Type of "z1" is partially unknown
+     Type of "z1" is "Unknown | Mul | Pow | Order | Expr | Add | None" (reportUnknownVariableType)
-     Type of "integrate" is "(..., meijerg: Unknown | None = None, conds: str = 'piecewise', risch: Unknown | None = None, heurisch: Unknown | None = None, manual: Unknown | None = None) -> (Unknown | Equality | Relational | Unequality)" (reportUnknownVariableType)
+     Type of "integrate" is "(..., meijerg: Unknown | None = None, conds: str = 'piecewise', risch: Unknown | None = None, heurisch: Unknown | None = None, manual: Unknown | None = None) -> (Unknown | Equality)" (reportUnknownVariableType)
-     Type of "t" is "Relational | Equality | Unequality | Unknown" (reportUnknownVariableType)
+     Type of "t" is "Relational | Equality | Unknown | Unequality" (reportUnknownVariableType)
-     Type of "expr" is "Relational | Equality | Unequality | Unknown" (reportUnknownVariableType)
+     Type of "expr" is "Relational | Equality | Unknown | Unequality" (reportUnknownVariableType)
-     Type of "expr" is "Relational | Equality | Unequality | Unknown" (reportUnknownVariableType)
+     Type of "expr" is "Relational | Equality | Unknown | Unequality" (reportUnknownVariableType)
-     Type of "expr" is "Relational | Equality | Unequality | Unknown" (reportUnknownVariableType)
+     Type of "expr" is "Relational | Equality | Unknown | Unequality" (reportUnknownVariableType)
-     Type of "i" is "Relational | Equality | Unequality | Unknown" (reportUnknownVariableType)
+     Type of "i" is "Relational | Equality | Unknown | Unequality" (reportUnknownVariableType)
-     Type of "i" is "Equality | Unknown | Relational | Unequality" (reportUnknownVariableType)
+     Type of "i" is "Equality | Unknown" (reportUnknownVariableType)
-     Type of "integrate" is "(..., meijerg: Unknown | None = None, conds: str = 'piecewise', risch: Unknown | None = None, heurisch: Unknown | None = None, manual: Unknown | None = None) -> (Unknown | Equality | Relational | Unequality)" (reportUnknownVariableType)
+     Type of "integrate" is "(..., meijerg: Unknown | None = None, conds: str = 'piecewise', risch: Unknown | None = None, heurisch: Unknown | None = None, manual: Unknown | None = None) -> (Unknown | Equality)" (reportUnknownVariableType)
-     Type of "pde" is "Equality | Unknown | Relational | Unequality" (reportUnknownVariableType)
+     Type of "pde" is "Equality | Unknown" (reportUnknownVariableType)
-     Type of "sol" is "Equality | Unknown | Relational | Unequality" (reportUnknownVariableType)
+     Type of "sol" is "Equality | Unknown" (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:465:5 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownParameterType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:465:5 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownParameterType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:536:12 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:536:12 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:539:5 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownParameterType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:539:5 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownParameterType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:634:12 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:634:12 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:638:5 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownParameterType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:638:5 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownParameterType)
-     Type of "tsol" is "Unknown | Equality | Relational | Unequality" (reportUnknownVariableType)
+     Type of "tsol" is "Unknown | Equality" (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:716:28 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:716:28 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownVariableType)
-     Type of "tsol" is "Unknown | Equality | Relational | Unequality" (reportUnknownVariableType)
+     Type of "tsol" is "Unknown | Equality" (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:724:28 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:724:28 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:733:16 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:733:16 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:742:16 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:742:16 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:760:16 - error: Return type, "Equality | Unknown | Relational | Unequality", is partially unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/pde.py:760:16 - error: Return type, "Equality | Unknown", is partially unknown (reportUnknownVariableType)
-     Type of "product" is "(...) -> (tuple[Unknown, ...] | Unknown | Product | Basic | Order | Any | Equality | Relational | Unequality)" (reportUnknownVariableType)
+     Type of "product" is "(...) -> (tuple[Unknown, ...] | Unknown | Product | Basic | Order | Any | Equality)" (reportUnknownVariableType)
-     Type of "K" is "tuple[Unknown, ...] | Unknown | Product | Basic | Order | Any | Equality | Relational | Unequality" (reportUnknownVariableType)
+     Type of "K" is "tuple[Unknown, ...] | Unknown | Product | Basic | Order | Any | Equality" (reportUnknownVariableType)
-     Type of "expand" is "Unknown | ((deep: bool = True, modulus: Unknown | None = None, power_base: bool = True, power_exp: bool = True, mul: bool = True, log: bool = True, multinomial: bool = True, basic: bool = True, **hints: Unknown) -> (Unknown | Order | Any))" (reportUnknownMemberType)
+     Type of "expand" is "Unknown | ((deep: bool = True, modulus: Unknown | None = None, power_base: bool = True, power_exp: bool = True, mul: bool = True, log: bool = True, multinomial: bool = True, basic: bool = True, **hints: Unknown) -> (Unknown | Order | Any | Self@Expr))" (reportUnknownMemberType)
-     Type of "i_part" is "Unknown | Order | Any" (reportUnknownVariableType)
+     Type of "i_part" is "Unknown | Order | Any | Self@Expr" (reportUnknownVariableType)
-     Type of "expand" is "Unknown | ((deep: bool = True, modulus: Unknown | None = None, power_base: bool = True, power_exp: bool = True, mul: bool = True, log: bool = True, multinomial: bool = True, basic: bool = True, **hints: Unknown) -> (Unknown | Order | Any)) | Any" (reportUnknownMemberType)
+     Type of "expand" is "Unknown | ((deep: bool = True, modulus: Unknown | None = None, power_base: bool = True, power_exp: bool = True, mul: bool = True, log: bool = True, multinomial: bool = True, basic: bool = True, **hints: Unknown) -> (Unknown | Order | Any | Self@Expr)) | Any" (reportUnknownMemberType)
-     Type of "ivl" is "Any | Unknown" (reportUnknownVariableType)
+     Type of "ivl" is "Unknown | Any" (reportUnknownVariableType)
-     Type of "as_set" is "Any | Unknown" (reportUnknownMemberType)
+     Type of "as_set" is "Any | (() -> Unknown) | Unknown" (reportUnknownMemberType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/solvers.py:2788:21 - error: Type of "b" is unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/solvers.py:2788:21 - error: Type of "b" is partially unknown
+     Type of "b" is "Mul | Unknown | Integer | Expr" (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/solvers.py:2788:24 - error: Type of "e" is unknown (reportUnknownVariableType)
+   /tmp/mypy_primer/projects/sympy/sympy/solvers/solvers.py:2788:24 - error: Type of "e" is partially unknown
+     Type of "e" is "Unknown | Expr | Mul | None" (reportUnknownVariableType)
-   /tmp/mypy_primer/projects/sympy/sympy/solvers/solvers.py:2788:28 - error: Type of "as_base_exp" is unknown (reportUnknownMemberType)

... (truncated 4247 lines) ...

spack (https://github.com/spack/spack)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-constrained-dependencies/package.py:20:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-constrained-dependencies/package.py:20:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-provider/package.py:22:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-provider/package.py:22:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-virtual-dependency/package.py:18:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-virtual-dependency/package.py:19:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-virtual-dependency/package.py:18:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditional-virtual-dependency/package.py:19:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditionally-patch-dependency/package.py:19:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conditionally-patch-dependency/package.py:19:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conflict-parent/package.py:17:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conflict-parent/package.py:17:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conflicting-dependent/package.py:19:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/conflicting-dependent/package.py:19:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/corge/package.py:20:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/corge/package.py:20:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/cumulative-vrange-middle/package.py:19:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/cumulative-vrange-middle/package.py:19:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/cumulative-vrange-root/package.py:19:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/cumulative-vrange-root/package.py:19:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/cumulative-vrange-root/package.py:20:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/cumulative-vrange-root/package.py:20:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-mid1/package.py:27:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-mid1/package.py:27:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-mid2/package.py:27:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-mid2/package.py:27:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-top/package.py:27:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-top/package.py:28:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-top/package.py:29:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-top/package.py:27:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-top/package.py:28:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-diamond-patch-top/package.py:29:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-with-variants-if-develop/package.py:16:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
+   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-with-variants-if-develop/package.py:16:5 - error: Result of call expression is of type "(pkg: PackageBase) -> None" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
-   /tmp/mypy_primer/projects/spack/var/spack/repos/builtin.mock/packages/dep-with-variants-if-develop-root/package.py:15:5 - error: Result of call expression is of type "((pkg: PackageBase) -> None)" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)

... (truncated 137 lines) ...

@DetachHead DetachHead merged commit 04163ee into main Sep 29, 2024
19 checks passed
@DetachHead DetachHead deleted the improve-baseline-matching branch September 29, 2024 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KotlinIsland KotlinIsland Awaiting requested review from KotlinIsland

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@DetachHead