Infer Protocols for function arguments without type hints

[diepala]

One point where VSCode lacks behind in python development when compared to other code editors/IDEs is in the ability to detect more errors by statically analyzing the code.

Take this example of code:

def compute_mean(vector):
    n = len(vector)
    return sum(vector.values) / n

mean = compute_mean([1, 2, 3])

The last line is incorrect as we are passing a list to the function, but it is not compatible with how the function works.
Because the function is unnanotated, VSCode does not report any error. However other IDEs (e.g. pycharm) can detect that something is wrong without having to run the code.

Could VS Code have something like this? It would really take the editor experience with python to the next level, even when writing unnanotated code.

Here there is how it could be done (and how I think other IDEs work). The main idea is to infer a Protocol for each unnanotated function argument:

1. Scan the function body to detect which attributes of the argument are being accessed.

In this case we can infer that the argument 'vector' must have an attribute called values and must define the function __len__.

2. Use the information to define a Protocol behind the scenes:

class VectorProtocol(Protocol):
    values: Iterable = ...
    def __len__(self) -> int:
        ...

3. Infer that the type of the arguments in the function are the Protocols defined previously:

The function definition would be equivalent to:

def compute_mean(vector: VectorProtocol):
    n = len(vector)
    return sum(vector.values) / n

Now, VSCode could give very useful errors when calling compute_mean([1, 2, 3]), as it would detect that the passed argument is incorrect because a list does not have any attribute values.

TL;DR:
When defining an unnanotated function like the following:

def compute_mean(vector):
    n = len(vector)
    return sum(vector.values) / n

VSCode could automatically detect it as the following piece of code behind the scenes:

class VectorProtocol(Protocol):
    values: Iterable = ...
    def __len__(self) -> int:
        ...

def compute_mean(vector: VectorProtocol):
    n = len(vector)
    return sum(vector.values) / n

This would help users catch errors earlier even if the code is not annotated. This functionality is present in other IDEs such as pycharm.

[erictraut]

This is an interesting idea. It's worth considering.

Pyright, the static type analyzer that underlies pyright, does already perform some advanced type inference for unannotated functions. In particular, it performs call-site inference in an attempt to infer return types. That functionality is intended primarily for users who are interested in more accurate completion suggestions but are not interested in bug detection.

It sounds like you are interested in pylance helping you find bugs in your code. Do you have type checking diagnostics enabled? That is, have you set "python.analysis.typeCheckingMode" to "basic"?

Pyright does not currently infer the types of unannotated parameters except for override methods where the base class provides type annotations. What you're suggesting would go beyond that and infer protocol types based on clues within the function's implementation. I'm not aware of other static type checking tools that do this. I'm quite certain that mypy, pyre, and pytype do not do this. Nor does TypeScript do this for unannotated javascript code. You mentioned that PyCharm's type checker does this. Could you point me to documentation for this feature? I'm skeptical that PyCharm is doing what you're suggesting, but it is possible that they're doing something different that is able to catch certain classes of bugs that pyright does not currently. Does this feature in PyCharm catch the bug in your specific code sample?

One challenge I see with your proposed solution is that it would be difficult or impossible to generate intelligible and actionable error messages at the call site. If PyCharm is able to detect the bug in the code above, I'd be interested in what error message it outputs and what code location it reports the bug against.

A big challenge with this solution is generating an accurate protocol definition. Python has many behaviors that are difficult to discern from looking at a code snippet. For example, you said "we can infer that the argument 'vector' must have an attribute called values". That's not strictly true. The argument vector might have an attribute called values, but it alternatively might have a __getattr__ method. There are also many cases where a parameter will be passed as an argument to other functions or methods that have overloaded signatures, which means the type is difficult or impossible to infer from its usage.

So my concern with your proposed approach is that it would produce a lot of false positive errors, and the resulting error messages would be difficult to understand leaving the user with no idea how to address them. The proposal would also be very computationally expensive, so it would significantly increase analysis times — the time for providing completion suggestions during editing.

[diepala]

Hi @erictraut, thanks for your answer!

Yes, I do have "python.analysis.typeCheckingMode" set to "basic", so the editor is already reporting type errors.

I also don't know any other type checker with this feature, only the integrated type checker inside pycharm.
I haven't found any documentation about it, because as far as I know, pycharm's type checker is not distributed as a standalone CLI tool. It is integrated into the editor.

Pycharm is able to catch the error with my code. Check this screenshoot:

In the function's call, when hovering over the argument, the editor gives the error that a list does not have the argument values.

In fact, when hovering over the argument in the function parameters, we can see what attributes the IDE has inferred over the parameter:

As you can see, the IDE identifies that vector have values and __len__ attributes.

Note that the error Pycharm shows is very similar to what pyright can show if I define the protocol. In VS Code, I obtain the following:

I imagine that inferring the possible attributes of the parameters by scanning the function body can be difficult or slow, but anyway, thanks for considering this feature!

[erictraut]

Thanks for the additional details.

Did you need to enable a feature in PyCharm's settings, or is this functionality enabled by default in PyCharm?

I think this idea has merit. I'll give it some more thought.

[diepala]

This behavior is enabled by default in PyCharm.

[laundmo]

The argument vector might have an attribute called values, but it alternatively might have a getattr method.

I don't think this type of issue would come up. The source of truth is not the Protocol, but the list which is passed in. We can't say that all arguments must explicitly define .values but we can say that there isn't any way for the list to have the .values attribute. It will have to assume that if the argument defines a __getattr__ / __getattribute__, anything goes (unless those can be inferred/hinted to only act on certain literals, not sure).

Making a Protocol from the function to check against may be the wrong approach as it inherently sees the Protocol as the source of truth.