Need some brainstorming for making Completion tests pass with DOMCompletionEngine

[gayanper]

Background

I started to work on enabling CompletionTests with DOMCompletionEngine, the high level plan is

• Create a new Javac Suite

• Take on test at a time

• Improve the DOMCompletionEngine to make tests pass

• Repeat last 2 steps until we have all tests passing

Now the idea is to just add features in DOMEngine until we have tests pass which gives us a good standing group to refactor the DOMEngine to make sure it is maintainable as well.

[gayanper]

Now I started with the first test case which is org.eclipse.jdt.core.tests.model.CompletionTests, while looking at the expectations of assertions, I see we have to make all our a proposals based on Bindings rather than IJavaElement, specially the use cases where we search for types by prefix.

One way would be to resolve bindings for each type we find from type search by converting Types into Reference and resolve Bindings. For that We need to have a reference to CompilationUnitDeclaration which we are completion on to get the scope for binding resolution.

Before I start on this path I would like to see other suggestions which we can achieve the same.

@mickaelistria @datho7561 @snjeza @testforstephen Any ideas ?

[gayanper]

I added a poc MR with some changes for resolving type bindings as well. #653

I think we could try to a cache type bindings during a single completion run if we see that we repeatedly resolving the same bindings as we improve the DOMEngine.

I really need some input since even though I have works a lot in CompletionEngine issue fixing, there are lot I don't yet know as well. I would really like to get input from everyone may be from the members in jdt.core project who also has lot more experience in this area.

[datho7561]

One thing that I'll comment on is that we think that DOM-based completion won't be good with ECJ, because the parser that's used for building the IJavaElements is not as fault-tolerant as the one that's used for completion.

The completion version of the ECJ parser (CompletionParser) uses a strategy where it inserts a special node in the AST that indicates where the completion was triggered. It creates this special node by performing some string manipulation on the source file. It does this, because when completion is invoked, the Java document is almost always in an invalid state.

javac leaves invalid nodes in the AST tree. For example, if you have myString., it will create a "qualified name" node where the first part is "myString" and the second part is a placeholder name. As a result, we might be able to use a similar strategy to CompletionParser when working with javac in order to ensure we have the completion context right.

I don't know how much of what I've said is useful to you. I have worked with CompletionParser before when working on Java 21 preview features, but I haven't looked much at the completion engine itself.

[gayanper]

Yes, at the movement the myString. handled by looking at how the javac actually generated it. So if for example ECJ would not do the same, I think we will not be able to use this at all with ECJ. CompletionParser actually do lot of handling to build a good enough compiler AST and the surrounding context so that CompletionEngine can make good choice on completions to suggest.

I thought in the same way we could look back and forward for tokens in the raw code, may be by using javac parser component and read tokens around the completion offset to make a good judgment like CompletionParser does't at parsing phase.

[datho7561]

hmmm yeah I think that's a good idea