Syntax highlighting keeps breaking

[JustinSmith66]

Type: Bug

my syntax highlighting for my python code randomly breaks (see the first variable is just "grey" in the first screenshot)

it is seriously annoying. I have looked on many online sources (https://stackoverflow.com/questions/59775038/visual-studio-code-syntax-highlighting-not-working#:~:text=In%20my%20case%20I%20needed,as%20source%20for%20the%20problem.) and this keeps happening.

VS Code version: Code 1.81.1 (6c3e3db, 2023-08-09T22:22:42.175Z)
OS version: Windows_NT x64 10.0.22621
Modes:

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 x 2592)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	31.85GB (17.32GB free)
Process Argv	--crash-reporter-id de061326-4473-4e85-9d97-746bfb43069b
Screen Reader	no
VM	0%

Extensions (9)

Extension	Author (truncated)	Version
isort	ms-	2023.10.1
pylint	ms-	2023.6.0
python	ms-	2023.14.0
vscode-pylance	ms-	2023.8.30
jupyter	ms-	2023.7.1002162226
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.0.17
vscode-jupyter-cell-tags	ms-	0.1.8
vscode-jupyter-slideshow	ms-	0.1.5

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
vstes263:30335439
vscoreces:30445986
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
pythonvs932:30410667
vsclangdc:30486549
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30695312
azure-dev_surveyone:30548225
vsccc:30803844
3biah626:30602489
89544117:30613380
showlangstatbar:30737416
03d35959:30757346
pythonfmttext:30731395
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
gsofb:30804716
pythonnosmt12:30797651
pythonidxpt:30805730
pythonnoceb:30805159
dsvsc013:30795093
dsvsc014:30804076
diffeditorv1:30812748

[hediet]

Can you clarify what you mean with syntax highlighting breaking?

[JustinSmith66]

Can you clarify what you mean with syntax highlighting breaking?

Please see the screenshot. At the top the variable name is just in grey. I believe this issue is also present in the stack overflow page i linked

[hediet]

Can you inspect editor tokens in in the broken editor (search in command palette) and share a screenshot?

[JustinSmith66]

r tokens in in the broken editor (search in command palette) and

i am not sure how to do this? Please may you advise? I am trying to search "Editor tokens" in the command pallet? Sorry i am not familiar with this

[hediet]

Please put your cursor on the code that is not tokenized correctly and invoke this command:

You get this UI by pressing F1 in VS Code. Then please share a screenshot of the hover! It looks like this:

[JustinSmith66]

So i see now my problem has progressed to what usually happens. No syntax highlighting works anymore. In my screenshot, when i hover over a variable it just says "loading" and never progresses from there. Usually, what I have to do is completely restart VSCode which is seriously frustrating

[JustinSmith66]

Please put your cursor on the code that is not tokenized correctly and invoke this command:

You get this UI by pressing F1 in VS Code. Then please share a screenshot of the hover! It looks like this:

I restarted my VS code and then got this to appear. However i had to restart VSC and reopen the Python notebook completely

[hediet]

Did you use the Inspect command?

[JustinSmith66]

Did you use the Inspect command?

yes i did from my understanding

[hediet]

I don't think that this is a tokenization problem, maybe the extension resources cannot be loaded or something (in notebook land?) else is off.

[JustinSmith66]

I don't think that this is a tokenization problem, maybe the extension resources cannot be loaded or something else is off.

I have looked at my extensions and uninstalled all the ones i dont use and have the same extensions installed as a friends (whose VSC works perfect)

I have uninstalled VSC many times and uninstalled python, formatted my PC - all the basic debugging I could do I did already.

[rebornix]

@JustinSmith66 do you happen to have Pylance extension installed?

[JustinSmith66]

@JustinSmith66 do you happen to have Pylance extension installed?

Yes i do have that installed. The list of extensions i have is listed in the original question's post

[rebornix]

@JustinSmith66 I'm wondering if you can reproduce this issue if you have Pylance disabled. Currently there is an issue with Pylance microsoft/pylance-release#4742 which might contribute to this.

[JustinSmith66]

@JustinSmith66 I'm wondering if you can reproduce this issue if you have Pylance disabled. Currently there is an issue with Pylance microsoft/pylance-release#4742 which might contribute to this.

but pylance is purely for syntax highlighting so would disabling it not just ruin the whole syntax highlighting feature ?I presume rolling back a few pylance versions is the only solution. But at the moment that seems to be the solution other people are using (microsoft/pylance-release#4742 (comment))

[rebornix]

@JustinSmith66 VS Code offers basic syntax highlighting for Python code and Pylance offers richer ones, I'm trying to figure out if it's an extension issue or it's a VS Code core issue. From what I read it's the same as microsoft/pylance-release#4742, let's track it there.