Add some kind of complex script compile time performance benchmark to track performance regressions

[gokiburikin]

I recently switched from 4.2 over 4.3dev6 and have noticed that the time it takes to compile (or whatever is happening in GDScript) one file in my game in particular has gone up enough for me to notice every single stall I get while code completion runs. Pressing play while this script is focused in the script editor also takes much longer than other scripts.

GDScript code completion times have been a pain point for me ever since 4.0 and they don't seem to be improving at all to the point I can't tell if I'm the only one using it this way.

If there was a decent benchmark test case that tested a real world scenario of a file with a decent chunk of code referencing a game sized amount of different classes then it would at least be possible to track the performance change over time across commits.

Here's a video of the save test: all I do is spam save and see how slow the compiler is. Comparing 4.2 to 4.3dev6 on basically the same file in the same project between versions. Showing a very simple script, very static but large game data script, and a large core gameplay logic script that references in the game data script often (over 100 times.)

4.2

Godot_v4.2-stable_win64_2024-05-02_11-07-59.mp4

4.3

Godot_v4.3-dev6_win64_2024-05-02_11-08-26.mp4

I've started a similar discussion before (#7487) and opened an issue (godotengine/godot#80637) but seeing as the problem appears to only be getting worse I think having a way to automatically track it through commits would go a long way in helping mitigate it. In that discussion I used the code parse setting test. You can also see performance by just holding Ctrl and trying to hover over code. They all showcase the stall which happens pretty much anytime you stop typing long enough that code completion initiates.

As a side note, because mouse inputs are handled before keyboard inputs this often results in this annoying behaviour:

1. double click to select some code
2. stall begins
3. press ctrl + c
4. click where I want to paste
5. press ctrl + v
6. stall ends
7. click goes through deselecting the selected text
8. ctrl + c goes through copying the entire line
9. ctrl + v goes through pasting the entire line

At this point the code completion is getting slow enough that it is absolutely a negative impact when editing certain scripts. The obvious solution is to split my code up more, but it doesn't change the fact that it takes quite some time.

For reference: Godot v4.3.dev6 - Windows 10.0.19044 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 2060 (NVIDIA; 31.0.15.5161) - Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz (16 Threads)

[Calinou]

Unfortunately, this is pretty difficult to test in an automated manner. https://github.com/godotengine/godot-benchmarks is a Godot project that performs its release runs with export template binaries, so testing editor-only functionality isn't possible with it. (Debug runs use an editor binary, but debug runs are only half of the picture when it comes to CPU performance testing.)

All we can do for now is have a benchmark that procedurally creates a GDScript from a large string and assigns its source code (with source code that doesn't actually do anything when run, due to it being skipped with an early return or similar).

[dalexeev]

Note that when you save the script (Ctrl + S), this causes it to be reloaded and recompiled. While autocompletion should be a little faster since only static analysis is performed.

As for the performance degradation in 3.x, this may not be a regression, but a consequence of fixing several bugs with dependencies (before you had to restart the editor to reset the cache).

However, I noticed that the editor performs redundant actions, calling the script analysis several times in a row. Probably debugging this is lower hanging fruit than improving the performance of the GDScript parser/compiler.

[gokiburikin]

There aren't many ways to easily showcase the stutter I get when the code completion first triggers, but it is shorter than saving. Here's what it looks like if I set code completion idle parse delay to 0 to exacerbate the frequency of the code completion stall:

Godot_v4.3-dev6_win64_2024-05-03_01-36-13.mp4

This is the lag that occurs when holding ctrl which makes commands like ctrl + c and ctrl+ v quite painful.

For what it's worth, the editor also seems to aggressively reload / recompile the current script when changing editor settings.

Godot_v4.3-dev6_win64_2024-05-03_01-40-40.mp4

4.3 is the first release where restarting the editor to fix an outdated script doesn't seem to be necessary which is a great convenience. Honestly if that's the tradeoff then I'll take it, but it sounds like there may be some performance things that can be addressed.

Thanks for the reply and the commits!

ETA: The video above shows a low idle parse delay (which you can tell from the line numbers flickering color.) Below is what it looks like with a 2 second idle parse but 0.01 code completion delay.

Godot_v4.3-dev6_win64_2024-05-03_01-53-07.mp4