I'm a Syntax Maintainer: Add Cutting Syntax Highlighting

[jeff-hykin]

Howdy!

I maintain the C++, C, and Shell syntax for VS Code, along with many other syntax extensions (Docker, Prolog, Perl, Objective-C, etc)

Before I made some big changes, I switched my C++ repo to use GPLv3. Meaning VS Code can no longer include it with VS Code. At the time I did this so that the highlighting on VS Codium could be better than what was on VS Code.

I love the Theia project so I wanted to let you know, the team is welcome to pull from my repo directly to ship the latest C++ syntax highlighting. If there's ever a change on my end (like changing where the .tmLanguage is stored) I'll make sure to tell you guys before pushing to master.

I'd love to be able to say Theia has better highlighting than VS Code by default.

Also, if you guys are ever curious about using the tree sitter parser instead of Texate (or along side Textmate), I'd love to be a part of that work.

[planger]

Hey there,

Thanks for reaching out and for your enthusiasm! It’s great to see such dedication to open source software. I appreciate your offer to enhance the C/C++ syntax highlighting in Theia, which is a significant area of interest in our CDT Cloud community too.

I noticed that your extension is already available on OpenVSX, which allows users of Theia IDE to install it and benefit from the improved syntax highlighting already today. Could you clarify a bit more about what you mean by pulling from your repo directly? Are you suggesting we add your extension by default to the Theia IDE (theiaPlugins), or are you thinking about integrating it into the VS Code builtin extensions that Theia uses?

Looking forward to collaborating further and potentially making Theia’s syntax highlighting even better than VS Code by default!

[jeff-hykin]

My understanding is VS Code comes with a lot of pre-defined languages, each with their own textmate grammar. Not as extensions, but just a collection of .tmLanguage files.

When they were publishing a new VS Code Insiders release they would pull the latest tmLanguage file from my repo (and many others) and include it in their build. Or, again, that is at least my understanding.

I haven't taken a deep look at how Theia works; maybe everything needs to be an extension and every built-in language needs to be a preinstalled extension. In that case, yes, I would be suggesting to pre-install the better C++ extension so there is some form of C++ syntax highlighting out of the box. In terms of bloat/minimalism, My understanding is Language grammars are used on-demand, meaning even a 1000 of them shouldn't add any startup cost. And if theyre not parsing the active file, there shouldnt be any runtime cost either.

If, for any reason, you dont want to include the .tmLanguage file or the extension, thats perfectly fine. Honestly it means less maintaince for me haha. But it would be kind of a nice to say Theia has objectively better C++ highlighting out of the box.

Note: the latest version of the grammar is still availble for VS Code as an extension too. GPL only restricts them from bundling it with VS Code.

[planger]

Thanks for the fast response and the additional details!

In Theia, we indeed handle syntax highlighting only through extensions, including those for built-in languages. The syntax highlighting for built-in languages, which also includes cpp, is pulled from VS Code and published as extension under vscode-builtin-extensions. This extension is maintained and built at https://github.com/eclipse-theia/vscode-builtin-extensions.

On top of that, we can pre-install a set of extensions for Theia IDE (see current list, which includes the aforementioned syntax highlighting).

And finally, users can choose to install additional extensions at runtime.

Given these three layers, we could (1) integrate your syntax highlighting in the vscode-builtin-extensions. However, I think there are some license considerations, because we would pull from GPL code and publish it as EPL 2.0 code, which is prohibited afaik. Alternatively, we could (2) just pre-install your syntax highlighting extension. This might be possible, but this needs to be clarified (see project handbook). GPL is not compatible with EPL, but it would be a week dependency. Still, it would be preinstalled and not optional per se, so it might be problematic too. Having users install the dependency (3) is certainly always possible.

I'll bring this up with the team to discuss the feasibility of integrating your extension. In general, I'm all for integrating your extension, as it objectively improves the C/C++ syntax highlighting. However, we need to consider the license implications. I’ll keep you posted on our decision.

Thanks again for your willingness to work with us and for considering Theia’s enhancement!

[tsmaeder]

Just chiming in here from the perspective of Theia the framework and Theia the IDE: GPL is not generally considered compatible with the EPL 2, so shipping the GPL'ed syntax as part of Theia IDE does not seem possible, as far as this non-lawyer can tell. Whether it's as a compile-time extension or a run time plugin does not make a difference IMO, since we would still distribute the combined work. Having GPL 3-licensed content would IMO force the whole package to be GPL 3.
Theia the framework does not contain any C++, C, or Shell syntax. However, our adopters would be in the same bind as VS Code if they want to ship a product that is not GPL, IMO.
So while I would of course like to have improved syntax highlighting, I'm not sure how we can consume the content under GPL 3.

[planger]

@tsmaeder Agreed! I also think it is problematic for adopters of the Theia platform to include GPL code. However, I think we may need to distinguish between Theia platform adotpers and Theia IDE users. But still, as you say, shipping the GPL extension by default also for Theia IDE users seems to be problematic, given we are EPL (or MIT in case of Theia IDE specific code).

[jeff-hykin]

Thanks for all the feedback! I keep half-responding to this and then loose my draft.

I'm perfectly fine changing the license to whatever, I just want to help community driven projects.

That said, this is actually a pretty challenging problem. It might be worth a bigger discussion.

_

The trivial solution is for me to just redo the changes (or similar changes) on EPL. But then that also means Microsoft can just pull Theia and extract the C++ syntax out of it.

Coming up with a solution where Theia can use it, but VS Code cannot is going to be tough. The only scenario I can imagine would require a number of things

1. The extension approach that was mentioned here, where extensions can have a license that is (at least slightly) different from Theia's EPL
2. Then have the sytax extension be AGPL (or something similar) that is designed to be dual licensed. The second license would be one I could issue specifically for Theia that basically gives Theia an exception to the copy-left clause of AGPL. Meaning, Theia can distrubute it, but if someone created a fork of Theia, they would be limited to one of the following:
   • Fork Theia but exclude the syntax extension
   • License the project under copyleft (and include the extension)
   • Reach a specific agreement with me the same way Theia did, include the extension, and maintain a more permissive license.

I'm no legal expert so idk if thats even possible. Even if possible, allowing copyleft as part of Theia's default extensions might go against the spirit of Theia (and its market partners). Again I'm fine with more permissive licensing, I just want to find some way for my open source work to give community projects an advantage over other options.

Their may be an alternative that I'm unaware of. For example, something like MIT for first-degree usage (direct fork of the original), but strong copyleft for second-degree usage (fork of a fork).