Enhancement: Syntax highlight for more programming languages

[zyxir]

What features would you like to see added?

Make syntax highlighting customizable for a wider range of programming languages.

More details

I have deployed LibreChat on my server and it has transformed my GPT and Gemini experience! The UI is sleek, the UX seamless, and the documentation has been a lifesaver. But as someone who works extensively with some less popular languages like Scala and Verilog, I've hit a snag—these languages aren't currently supported for syntax highlighting, despite being supported by highlight.js.

While this intentional behavior aligns with ChatGPT, it restricts customizability within LibreChat as a deployable web application. Integrating configurable highlighting, even through code edits, would be a huge advantage over the default ChatGPT UI.

Additional infomation:

1. I directly added languages to the langSubset constant in client/src/utils/languages.ts, but the changes didn't produce the expected highlighting behavior.
2. With Google Bard already highlighting Scala and Verilog, including them as optional features in LibreChat would go a long way towards enhancing the user experience for developers like myself. I believe incorporating this change would attract a wider audience and solidify LibreChat's position as a powerful and versatile tool for code interaction.

Which components are impacted by your request?

UI

Pictures

Bard:

ChatGPT:

LibreChat:

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Thanks for bringing this to my attention.

Unfortunately, having more languages can seriously affect performance. Though I did learn how to correctly configure this in the process in case you would like to customize this.

The following all done in client/src/components/Chat/Messages/Content/Markdown.tsx, and langSubset edits in client/src/utils/languages.ts

You need to import the language you want if not already highlighted, should already be installed. I believe default is using "common" languages according to rehype-highlight docs, and they still seem to be included even when adding more languages like I did below.

Note: highlight.js should already be installed

import scala from 'highlight.js/lib/languages/scala';

// then change this relevant part as follows
  const rehypePlugins: PluggableList = [
    [rehypeKatex, { output: 'mathml' }],
    [
      rehypeHighlight,
      {
        detect: true,
        ignoreMissing: true,
        languages: { scala },
        subset: langSubset,
      },
    ],
    [rehypeRaw],
  ];

I directly added languages to the langSubset constant in client/src/utils/languages.ts, but the changes didn't produce the expected highlighting behavior.

subset option indeed does not seem to do what I thought, since including scala within langSubset didn't seem to affect highlighting when i included scala in languages.

I would still include it since it's a real option according to the docs:

const langSubset = [
  'python',
// omitted
  'scala',
];

And in case you want to ignore some of the "common" languages, this works for me

rehype-highlight docs: https://github.com/rehypejs/rehype-highlight#options

      rehypeHighlight,
      {
        detect: true,
        ignoreMissing: true,
        languages: { scala },
        plainText: ['python'], // no longer highlights python
        subset: langSubset,
      },

[zyxir]

Thank you for your timely response. Unfortunately, after added the code as you suggested, the web UI has malfunctioned and no longer displays the login fields.

The "欢迎" here means "welcome" in my language. It is worth mentioning that I am using the native version without a docker container. I tried npm run frontend, but the problem persisted. I also tried restarting the server, same result. I noticed #1445 describes a similiar problem, but the solution applies to the docker version, which I am not using.

Even after I removed those code edits, the problem persisted. I wonder if I can reclone and rebuild the repository without data loss.

[fuegovic]

Did you try accessing the website from a private/incognito browser session?

If you use the online database, as long as you use the same connection string you will not loose any data.

[zyxir]

Yes, I tried, and it did not solve the problem.

[zyxir]

I'm so sorry for the false alarm! It turns out the UI issue was entirely my fault due to accidentally turning off email login. I feel terrible for wasting your time, but I'm grateful for your patience and the helpful code provided by @danny-avila. Syntax highlighting is now working perfectly. I've learned a valuable lesson about double-checking my settings before reporting issues!

While I have your attention, I'd also like to respectfully inquire about the possibility of enhancing syntax highlighting customization. Would it be feasible to consider adding options or environment variables to enable users to more easily add support for additional languages, without requiring direct code modification? I understand adding this feature might require more experienced hands, but I'd be honored to learn some JavaScript (or TypeScript) and create a pull request.

[danny-avila]

While I have your attention, I'd also like to respectfully inquire about the possibility of enhancing syntax highlighting customization. Would it be feasible to consider adding options or environment variables to enable users to more easily add support for additional languages, without requiring direct code modification? I understand adding this feature might require more experienced hands, but I'd be honored to learn some JavaScript (or TypeScript) and create a pull request.

Yes that could be a good feature. For now I'm prioritizing some things already on the roadmap, such as a config yaml file, which may in turn help this cause

[zyxir]

The use of YAML for configuration is brilliant. I've tried over 10 LLM UIs, and LibreChat is the only one that meets all of my needs. It's powerful, flexible, and easy to use. Thank you for creating such a great product. I'm really grateful for all of your hard work.

[danny-avila]

@zyxir I appreciate your feedback and glad it serves you well!

[terrabys]

How bad is syntax highlighting performance hit?
I ran a prompt to generate 'Hello World' in 50 different languages, with LibreChat and your competitors,
I didn't noticed a huge difference in performance with syntax highlighting for more languages.

Although it's not a good test, because these specific competitors have overall snappier and more responsive feeling UI

Maybe there should be a guide in the docs on how to add highlighting to more languages.