Allow assigning ellipsis literal as parameter default value

[Glyphack]

Resolves #14840

Summary

Usage of ellipsis literal as default argument is allowed in stub files.

Test Plan

Added mdtest for both python files and stub files.

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

[AlexWaygood]

The pre-commit failure will go away if you merge in main or rebase :-)

[Glyphack]

Wow there's a bug in the new github UI for PRs it was not showing the rebase button I thought it's already up to date.

[AlexWaygood]

Wow there's a bug in the new github UI for PRs it was not showing the rebase button I thought it's already up to date.

Oh, that only appears for a repository if the maintainers have checked a certain box in the repository settings on GitHub. We haven't enabled it for Astral repositories because the rate of development is quite rapid, and some folks found it annoying to be constantly told that all their PRs were out of date with the main branch 😆

[carljm]

Thank you!!

[Glyphack]

Thanks for the review.
I have a question, pyright behavior here is that in all files the assignment works only if the function body is also ellipsis. The link does not specify if it's stub file but pasting this into a python file locally shows the same warnings.
https://pyright-play.net/?strict=true&code=CYUwZgBGAUAeBcECWA7ALhAvBAdHglBALQB8EAcgPYojwBQEjEATiGgK7MoSx12iQYCZOiy4CxMlRqI8OIA
Mypy does the same. Is this just a limitation of them or the specs are not strictly forbidding this?

[AlexWaygood]

This special case is only allowed in function and method arguments. Based on what I understand from typing.readthedocs.io/en/latest/guides/writing_stubs.html#module-level-attributes and typing.readthedocs.io/en/latest/guides/writing_stubs.html#classes.

Do not unnecessarily use an assignment for module-level attributes.

Ah, I think there's a small misunderstanding here. There are two halves of typing.readthedocs.io:

• One half of the website (https://typing.readthedocs.io/en/latest/spec/) is a formal typing spec: this is a set of rules that type checkers must follow if they claim to support all features of Python's typing system
• The other half of the website (https://typing.readthedocs.io/en/latest/guides/) is a set of guides for users on how they should use annotations in their own runtime code and stub files. These are guides to best practices, but they aren't necessarily things that type checkers need to enforce

The two pages you link to there are both in the "second half" of the website: those paragraphs describe best practices for people writing stub files, but these aren't necessarily rules that type checkers have to enforce regarding stub files. The typing spec part of typing.readthedocs.io is here: https://typing.readthedocs.io/en/latest/spec/distributing.html#value-expressions. It states that type checkers should allow ... as a value in many more places, even if the "guides" part of typing.readthedocs.io states that it wouldn't necessarily be best practice for users to do so. Here's what the spec has to say:

In locations where value expressions can appear, such as the right-hand side of assignment statements and function parameter defaults, type checkers should support the following expressions:

• The ellipsis literal, ..., which can stand in for any value
  -Any value that is a legal parameter for typing.Literal
• Floating point literals, such as 3.14
• Complex literals, such as 1 + 2j

And later:

Type checkers should support module-level variable annotations, with and without assignments:

x: int
x: int = 0
x = 0  # type: int
x = ...  # type: int

[AlexWaygood]

So I think we should allow ... as a value for any annotated symbol in stub files, in any context, not just parameter defaults

[carljm]

Thanks for that clarification, Alex! I was surprised that x: int = ... wouldn't be supported in a stub, and I read the links without noticing they weren't part of the spec.

I feel like the wording of the spec:

In locations where value expressions can appear... type checkers should support the following expressions: ... The ellipsis literal, ..., which can stand in for any value

Suggests a totally different implementation approach here, which is that in a stub file we should simply infer literal ... expressions as type Any, which is assignable to anything. That seems better than trying to handle this at each point where we enforce assignability.

[carljm]

I have a question, pyright behavior here is that in all files the assignment works only if the function body is also ellipsis ... Mypy does the same

That's very interesting, good observation! I don't actually know why that is. It may be in order to support ellipsis defaults in overloads and protocol methods. I'm a little surprised that this appears to be handled by allowing it based on the body being ..., rather than just allowing it contextually for overloads and protocols.

I'm not convinced we want to follow suit here; it seems incorrect to do this based on a body of ..., considering that is a valid body for a regular Python function.

So I would say let's implement this only for stubs for now, and consider the best approach when we add overloads/protocols support.

[AlexWaygood]

Suggests a totally different implementation approach here, which is that in a stub file we should simply infer literal ... expressions as type Any, which is assignable to anything. That seems better than trying to handle this at each point where we enforce assignability.

That seems okay to me as long as we don't start inferring unions with Any for things imported from stub files... but I don't think we will, because declared types take precedence over inferred types for things imported from other modules, right?

[carljm]

That seems okay to me as long as we don't start inferring unions with Any for things imported from stub files... but I don't think we will, because declared types take precedence over inferred types for things imported from other modules, right?

Yes, that's right.

[Glyphack]

Thanks let me do try to change this to infer the type of ellipsis literal as Any instead of adding this special case.

[carljm]

Suggests a totally different implementation approach here, which is that in a stub file we should simply infer literal ... expressions as type Any, which is assignable to anything. That seems better than trying to handle this at each point where we enforce assignability.

Ok, after further discussion (thanks @AlexWaygood!) I think this is not a good idea, and might cause undesirable behavior in examples like this:

class Foo: ...

X: int = ...
Y: Foo = X

Where this should be flagged as an error, even in a type stub, but my suggestion (combined with our current handling of inferred dynamic type) would not consider it an error. (It might be worth including this as a test case.)

So I think the better approach is in fact to special-case handling of literal ... in three different places: in function parameters, where you're doing it now; in infer_annotated_assignment_definition, and in infer_assignment_definition.

In the first two cases, if we see a literal ... as the default-value or RHS, in a stub file, we should just discard that inference and record the declared type as the inferred type. In the third case, there is no declaratiion and we should record Unknown.

[carljm]

Thank you! A few comments.

[carljm]

We should use Type::Unknown rather than Type::Any here. In our model, Type::Any is reserved for explicit uses of typing.Any as an annotation. Any dynamic type arising from lack of an annotation uses Type::Unknown.

Also we should add a test for this case (un-annotated assignment in a stub file, with ... as RHS.)

[carljm]

It would be unusual, but I think valid according to the spec to do FOO, BAR = ... in a stub file. So I think we also should add a special case for ... (only if file.is_stub(db) of course) in infer_unpack_types (where we infer the value_ty from the RHS expression). And a test for this.

[Glyphack]

I applied this change inside the unpacker since that was the first place that value type was being inferred. Also there was a match to check the expression being used. I hope this is what you meant as well.

But if I understand correctly we don't want the unpacking to happen in for loops so I'm checking that unpack value is of type UnpackerValue::Assign.

[Glyphack]

I spent some time to see if I can pull out the code for special case to infer_unpack_types itself. But I think we want the whole unpacking behavior to happen.
This could be done if we only want to treat the case where LHS is a tuple and RHS is .... But I feel this behavior would not be consistent. I also checked Pyright.

For the following .pyi file Pyright infers all variables the same.

a, b = ...
reveal_type(a)
reveal_type(b)

[c] = ...
reveal_type(c)

I can't decide if we should only allow a, b = ... or other forms of unpacking as well so please let me know if you think the other way is better.

[carljm]

It seems we've forgotten the condition "and the value expression is a literal ..." here. If that doesn't fail any test, we should definitely add a test with e.g. x: int = 1 in a stub file showing we still infer Literal[1] for x, not Unknown.

[Glyphack]

Yes it was not failing. Added a new test for it in mdtest/assignment/annotations.md.

[MichaReiser]

Nit: It might be worth introducing a is_stub function on InferenceBuilder that does the self.file().is_stub(self.db().upcast())

[Glyphack]

Out of curiosity I checked what pyright and mypy do when they encounter ... while unpacking RHS(this comment) in assignment.

With this test.pyi(in .py files this is invalid):

x, y = ...

Pyright:

0 errors, 0 warnings, 0 informations

Mypy:

test.pyi:1: error: "EllipsisType" object is not iterable  [misc]
Found 1 error in 1 file (checked 1 source file)

[Glyphack]

I wanted to keep this consistent with is_iterable so I created this. I think https://docs.rs/is-macro/latest/is_macro/ also works here.