Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development: Upgrade markdown library to markdown-it #9354

Merged
merged 45 commits into from
Oct 27, 2024

Conversation

Strohgelaender
Copy link
Contributor

@Strohgelaender Strohgelaender commented Sep 22, 2024

Checklist

General

Client

  • Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data (e.g. using paging).
  • I strictly followed the principle of data economy for all client-server REST calls.
  • I strictly followed the client coding and design guidelines.
  • Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
  • I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
  • I documented the TypeScript code using JSDoc style.
  • I added multiple screenshots/screencasts of my UI changes.

Motivation and Context

The currently used showdown library for markdown parsing is unmaintained, see #9333.
This PR replaces it with the more modern library markdown-it.

Closes #9333.

Description

These dependencies were added:

  • markdown-it: Replacement for showdown (Markdown -> HTML)
  • turndown: Replacement for showdown (HTML -> Markdown)
  • markdown-it-katex: Replacement for showdown-katex.
  • markdown-it-highlightjs: Replacement for showdown-highlight
  • markdown-it-class Replaces the custom extension to improve table styling.

I migrated the Task and PlantUML extensions to MarkdownIt Plugins and added a custom one that ensues backwards compatibility for inline formulas.

This fixes issue #768, but formulas with escaped underscores now don't work. I did not find a away to ensure compatibility here, feel free to suggest something.

Steps for Testing

Please test the different places where Markdown is rendered, especially:

  • Programming Exercises with Tasks and PlantUML diagrams
  • Exercises with Tables and Katex formulas

I created a small exercise on TS2 that includes these special cases: https://artemis-test2.artemis.cit.tum.de/course-management/10/programming-exercises/283

Deploy this and another PR and check that the Problem Statement renders correctly.

Also test quiz exercises, especially short answer quizzes, here the markdown parsing has some additional complexity.

Exam Mode Testing

Please test that problem statements are correctly rendered when taking an exam. Please test both the online code editor and the exercise view (when only offline editors are activated). Also test quiz exercises of the different types.

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.







Review Progress

Performance Review

  • I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance even for very large courses with more than 2000 students.
  • I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance even for very large courses with more than 2000 students.

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Exam Mode Test

  • Test 1
  • Test 2

Performance Tests

  • Test 1
  • Test 2

Test Coverage

Screenshots

Summary by CodeRabbit

Release Notes

  • New Features

    • Transitioned from Showdown to Markdown-it for enhanced markdown processing.
    • Introduced new plugins for LaTeX rendering and syntax highlighting.
  • Bug Fixes

    • Updated HTML rendering expectations in tests for consistency.
  • Documentation

    • Revised documentation to reflect changes in markdown processing and updated links to resources.
  • Chores

    • Removed outdated dependencies and added new ones to support the Markdown-it library.
  • Tests

    • Enhanced test coverage for LaTeX formula processing in markdown.

@github-actions github-actions bot added the client Pull requests that update TypeScript code. (Added Automatically!) label Sep 22, 2024
@github-actions github-actions bot added the tests label Sep 22, 2024
@Strohgelaender Strohgelaender marked this pull request as ready for review September 22, 2024 19:58
@Strohgelaender Strohgelaender requested a review from a team as a code owner September 22, 2024 19:58
Copy link

coderabbitai bot commented Sep 22, 2024

Walkthrough

The pull request introduces significant changes to the markdown processing within the project by replacing the Showdown library with Markdown-it and its associated plugins. Modifications include updates to configuration files, dependency management, and the implementation of new classes and methods. The changes streamline the handling of markdown, particularly with regards to LaTeX support and text replacement functionalities, while also addressing vulnerabilities associated with the Showdown library.

Changes

File Change Summary
angular.json Removed showdown, showdown-highlight, showdown-katex; added markdown-it-class.
package.json Added @vscode/markdown-it-katex, markdown-it, markdown-it-class, markdown-it-highlightjs; removed showdown, showdown-highlight, showdown-katex, katex, and type definitions for @types/showdown. Added type definitions for @types/markdown-it, @types/turndown.
src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-plant-uml.extension.ts Changed class inheritance from ArtemisShowdownExtensionWrapper to ArtemisTextReplacementPlugin; updated method from getExtension() to replaceText(text: string).
src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-task.extension.ts Changed class inheritance from ArtemisShowdownExtensionWrapper to ArtemisTextReplacementPlugin; removed getExtension(); added replaceText(text: string).
src/main/webapp/app/exercises/programming/shared/instructions-render/programming-exercise-instruction.component.ts Updated markdownExtensions type from ShowdownExtension[] to PluginSimple[].
src/main/webapp/app/shared/markdown-editor/extensions/ArtemisTextReplacementPlugin.ts Added new abstract class ArtemisTextReplacementPlugin with methods getExtension() and replaceText(text: string).
src/main/webapp/app/shared/markdown.service.ts Updated safeHtmlForMarkdown method to use PluginSimple[] instead of showdown.ShowdownExtension[].
src/main/webapp/app/shared/util/markdown.conversion.util.ts Refactored to use Markdown-it; updated htmlForMarkdown and markdownForHtml functions.
src/test/javascript/spec/component/lecture-unit/text-unit/text-unit.component.spec.ts Updated exampleHtml constant from <h3> to <h1>.
src/test/javascript/spec/component/shared/metis/posting-content/posting-content-part.component.spec.ts Removed trailing space in expected HTML output.
src/test/javascript/spec/service/markdown.service.spec.ts Added tests for formulaCompatibilityPlugin in Markdown Service.
docs/user/markdown-support.rst Updated documentation links and references from Showdown to Markdown-it.
src/main/webapp/app/shared/markdown-editor/monaco/markdown-editor-monaco.component.html Updated hyperlink from Showdown demo to Markdown-it documentation.

Assessment against linked issues

Objective Addressed Explanation
Address vulnerability related to showdown-2.1.0 (9333)

Possibly related PRs

Suggested labels

server, bugfix

Suggested reviewers

  • JohannesStoehr
  • az108
  • pzdr7
  • krusche

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 1393b0d and 2a5acb6.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (2)
  • package.json (4 hunks)
  • src/main/webapp/app/shared/markdown-editor/monaco/markdown-editor-monaco.component.html (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • package.json
  • src/main/webapp/app/shared/markdown-editor/monaco/markdown-editor-monaco.component.html

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

Outside diff range comments (3)
angular.json (1)

Line range hint 1-287: Consider additional configuration changes for markdown-it integration.

While the current changes correctly add markdown-it-class to the allowed dependencies, there might be additional configuration changes needed for a complete migration from showdown to markdown-it.

Please consider the following:

  1. Review the scripts and styles sections to ensure any showdown-related entries are removed and markdown-it entries are added if necessary.
  2. Check if any fileReplacements or environment-specific configurations need updating to reflect the new markdown processing library.
  3. Verify that the budgets section accommodates any size changes resulting from the library switch.
  4. Ensure that any custom webpack configurations (if used) are updated to handle markdown-it and its plugins correctly.

These steps will help ensure a smooth and complete transition to the new markdown processing library throughout the Angular application.

src/main/webapp/app/exercises/quiz/shared/short-answer-question-util.service.ts (1)

Line range hint 1-458: Consider refactoring for improved maintainability

While the immediate change is beneficial, there are opportunities for broader improvements in this file:

  1. Some methods, like addIndentationToTextParts, are quite long and complex. Consider breaking them down into smaller, more focused methods to improve readability and maintainability.

  2. There are several TODO comments throughout the code. It would be beneficial to address these or create issues to track them if they represent larger tasks.

  3. The ShortAnswerQuestionUtil class is quite large. Consider if some of its functionality could be split into separate, more focused service classes.

  4. Ensure all methods have appropriate JSDoc comments for better documentation.

These suggestions aim to improve the overall code quality and maintainability of the file.

src/main/webapp/app/exercises/programming/shared/instructions-render/programming-exercise-instruction.component.ts (1)

Issues Found: Remaining 'showdown' References in Codebase

The migration from showdown to markdown-it is not fully complete. The following files still contain references to showdown, which may lead to inconsistencies in markdown rendering:

  • src/main/webapp/app/shared/util/markdown.conversion.util.ts
    • Contains comments referencing showdown behavior.
  • src/main/webapp/app/shared/markdown-editor/monaco/markdown-editor-monaco.component.html
    • Includes a link to the showdown demo: <a class="col-auto" href="http://demo.showdownjs.com">

Please remove or update these lingering showdown references to ensure a complete and consistent transition to markdown-it.

Analysis chain

Line range hint 1-438: Verify markdown rendering functionality

While the changes to accommodate the new markdown-it library appear correct, it's crucial to thoroughly test the markdown rendering functionality across various scenarios. This includes:

  1. Rendering of basic markdown elements (headers, lists, code blocks, etc.)
  2. Proper functioning of custom extensions (e.g., task parsing, UML diagrams)
  3. Interaction with other components that consume rendered markdown

Please ensure that all existing markdown features work as expected with the new library implementation.

To assist in verifying the markdown rendering, you can run the following script to check for any potential issues or inconsistencies in the markdown processing:

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for potential issues in markdown rendering

# Test: Look for any remaining references to 'showdown' in the codebase
echo "Checking for remaining 'showdown' references:"
rg 'showdown' --type typescript --type html

# Test: Verify that all markdown-it plugins are properly imported and used
echo "Verifying markdown-it plugin usage:"
rg 'markdown-it-' --type typescript

# Test: Check for any TODO comments related to markdown processing
echo "Checking for TODO comments related to markdown:"
rg 'TODO.*markdown' --type typescript --type html

Length of output: 529


Script:

#!/bin/bash
# Description: Check for potential issues in markdown rendering

# Test: Look for any remaining references to 'showdown' in the codebase
echo "Checking for remaining 'showdown' references:"
rg 'showdown' --type ts --type html

# Test: Verify that all markdown-it plugins are properly imported and used
echo "Verifying markdown-it plugin usage:"
rg 'markdown-it-' --type ts

# Test: Check for any TODO comments related to markdown processing
echo "Checking for TODO comments related to markdown:"
rg 'TODO.*markdown' --type ts --type html

Length of output: 1527

Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between 00f38c0 and 6d7c856.

Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
Files selected for processing (16)
  • angular.json (1 hunks)
  • package.json (4 hunks)
  • src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-plant-uml.extension.ts (3 hunks)
  • src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-task.extension.ts (3 hunks)
  • src/main/webapp/app/exercises/programming/shared/instructions-render/programming-exercise-instruction.component.ts (2 hunks)
  • src/main/webapp/app/exercises/quiz/shared/short-answer-question-util.service.ts (1 hunks)
  • src/main/webapp/app/index.d.ts (1 hunks)
  • src/main/webapp/app/shared/markdown-editor/extensions/ArtemisTextReplacementExtension.ts (1 hunks)
  • src/main/webapp/app/shared/markdown-editor/extensions/artemis-showdown-extension-wrapper.ts (0 hunks)
  • src/main/webapp/app/shared/markdown.service.ts (2 hunks)
  • src/main/webapp/app/shared/pipes/html-for-markdown.pipe.ts (2 hunks)
  • src/main/webapp/app/shared/util/markdown.conversion.util.ts (3 hunks)
  • src/test/javascript/spec/component/lecture-unit/text-unit/text-unit.component.spec.ts (1 hunks)
  • src/test/javascript/spec/component/shared/metis/posting-content/posting-content-part.component.spec.ts (1 hunks)
  • src/test/javascript/spec/component/short-answer-quiz/short-answer-question-util.service.spec.ts (1 hunks)
  • src/test/javascript/spec/helpers/sample/problemStatement.json (1 hunks)
Files not reviewed due to no reviewable changes (1)
  • src/main/webapp/app/shared/markdown-editor/extensions/artemis-showdown-extension-wrapper.ts
Additional context used
Path-based instructions (12)
src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-plant-uml.extension.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-task.extension.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/exercises/programming/shared/instructions-render/programming-exercise-instruction.component.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/exercises/quiz/shared/short-answer-question-util.service.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/index.d.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/shared/markdown-editor/extensions/ArtemisTextReplacementExtension.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/shared/markdown.service.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/shared/pipes/html-for-markdown.pipe.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/main/webapp/app/shared/util/markdown.conversion.util.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/test/javascript/spec/component/lecture-unit/text-unit/text-unit.component.spec.ts (1)

Pattern src/test/javascript/spec/**/*.ts: jest: true; mock: NgMocks; bad_practices: avoid_full_module_import; perf_improvements: mock_irrelevant_deps; service_testing: mock_http_for_logic; no_schema: avoid_NO_ERRORS_SCHEMA; expectation_specificity: true; solutions: {boolean: toBeTrue/False, reference: toBe, existence: toBeNull/NotNull, undefined: toBeUndefined, class_obj: toContainEntries/toEqual, spy_calls: {not_called: not.toHaveBeenCalled, once: toHaveBeenCalledOnce, with_value: toHaveBeenCalledWith|toHaveBeenCalledExactlyOnceWith}}

src/test/javascript/spec/component/shared/metis/posting-content/posting-content-part.component.spec.ts (1)

Pattern src/test/javascript/spec/**/*.ts: jest: true; mock: NgMocks; bad_practices: avoid_full_module_import; perf_improvements: mock_irrelevant_deps; service_testing: mock_http_for_logic; no_schema: avoid_NO_ERRORS_SCHEMA; expectation_specificity: true; solutions: {boolean: toBeTrue/False, reference: toBe, existence: toBeNull/NotNull, undefined: toBeUndefined, class_obj: toContainEntries/toEqual, spy_calls: {not_called: not.toHaveBeenCalled, once: toHaveBeenCalledOnce, with_value: toHaveBeenCalledWith|toHaveBeenCalledExactlyOnceWith}}

src/test/javascript/spec/component/short-answer-quiz/short-answer-question-util.service.spec.ts (1)

Pattern src/test/javascript/spec/**/*.ts: jest: true; mock: NgMocks; bad_practices: avoid_full_module_import; perf_improvements: mock_irrelevant_deps; service_testing: mock_http_for_logic; no_schema: avoid_NO_ERRORS_SCHEMA; expectation_specificity: true; solutions: {boolean: toBeTrue/False, reference: toBe, existence: toBeNull/NotNull, undefined: toBeUndefined, class_obj: toContainEntries/toEqual, spy_calls: {not_called: not.toHaveBeenCalled, once: toHaveBeenCalledOnce, with_value: toHaveBeenCalledWith|toHaveBeenCalledExactlyOnceWith}}

Biome
src/test/javascript/spec/component/shared/metis/posting-content/posting-content-part.component.spec.ts

[error] 97-97: Forbidden non-null assertion.

Unsafe fix: Replace with optional chain operator ?. This operator includes runtime checks, so it is safer than the compile-only non-null assertion operator

(lint/style/noNonNullAssertion)

Additional comments not posted (28)
src/main/webapp/app/index.d.ts (1)

1-4: Verify the impact of removing showdown-related declarations.

The replacement of showdown-related declarations with the new markdown-it-class declaration aligns well with the PR objective and addresses the security vulnerability mentioned in issue #9333. However, this change may have broader implications across the codebase.

Please run the following script to ensure that all references to the removed showdown modules have been updated or removed:

This script will help identify any places where the old modules are still referenced and confirm that the new markdown-it-class module is being used correctly.

Verification successful

Verification Successful: Showdown modules correctly removed and markdown-it-class integrated.

All references to 'showdown-katex' and 'showdown-highlight' have been successfully removed, and 'markdown-it-class' is being utilized as intended in the codebase.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any remaining references to removed showdown modules

echo "Checking for references to showdown-katex:"
rg --type typescript --type javascript "showdown-katex"

echo "Checking for references to showdown-highlight:"
rg --type typescript --type javascript "showdown-highlight"

echo "Verifying markdown-it-class usage:"
rg --type typescript --type javascript "markdown-it-class"

Length of output: 556


Script:

#!/bin/bash
# Description: Check for any remaining references to removed showdown modules

echo "Checking for references to showdown-katex:"
rg --glob "*.ts" --glob "*.js" "showdown-katex"

echo "Checking for references to showdown-highlight:"
rg --glob "*.ts" --glob "*.js" "showdown-highlight"

echo "Verifying markdown-it-class usage:"
rg --glob "*.ts" --glob "*.js" "markdown-it-class"

Length of output: 604

src/main/webapp/app/shared/markdown-editor/extensions/ArtemisTextReplacementExtension.ts (5)

1-2: LGTM: Imports are correctly typed and follow best practices.

The imports are properly using the 'type' keyword for type-only imports, which is good for optimizing bundle size. The naming conventions follow TypeScript standards.


4-7: LGTM: Class declaration and documentation are well-structured.

The abstract class ArtemisTextReplacementExtension is properly declared and exported. The class name follows the PascalCase convention as per the coding guidelines. The JSDoc comment effectively describes the purpose of the class.


8-19: LGTM: getExtension() method is well-implemented.

The getExtension() method is correctly implemented as a MarkdownIt plugin. It effectively overrides the render method while preserving the original functionality. The use of arrow functions and the spread operator aligns with the coding guidelines. The method name follows the camelCase convention as required.


21-21: LGTM: replaceText() abstract method is properly declared.

The replaceText() method is correctly declared as abstract, following the camelCase naming convention. Its signature is clear and concise, with appropriate parameter and return types.


1-22: Great implementation of the Markdown-It extension for text replacement.

This new ArtemisTextReplacementExtension class aligns well with the PR objectives of upgrading the markdown library. It provides a flexible and reusable way to implement text replacement in raw markdown before processing. The implementation follows TypeScript and Angular coding guidelines, making it a valuable addition to the project.

Some key points:

  1. The class is designed to be extended, allowing for various text replacement strategies.
  2. It correctly integrates with Markdown-It by providing a plugin that overrides the render method.
  3. The original rendering functionality is preserved while adding the new text replacement step.

This implementation contributes to the overall goal of replacing the showdown library with markdown-it, addressing the security concerns mentioned in the linked issue #9333.

src/main/webapp/app/shared/pipes/html-for-markdown.pipe.ts (4)

15-15: LGTM: Method signature comment updated correctly.

The JSDoc comment has been updated to reflect the change from ShowdownExtension[] to PluginSimple[], which is consistent with the upgrade to markdown-it. The comment follows the project's coding guidelines for documentation.


Line range hint 1-28: Overall assessment: Changes successfully implement markdown-it upgrade.

The modifications in this file effectively transition from the showdown library to markdown-it, aligning with the PR objectives. The changes are consistent throughout the file, including imports, JSDoc comments, and method signatures. The implementation adheres to the project's coding guidelines and maintains the existing functionality while upgrading the markdown processing library.


4-4: LGTM: Import statement for markdown-it added correctly.

The import of PluginSimple from 'markdown-it' is in line with the PR objective of upgrading to the markdown-it library. This change is correctly implemented and follows the project's coding guidelines.

Let's verify the usage of PluginSimple in the file:

#!/bin/bash
# Description: Verify the usage of PluginSimple in the file.

# Test: Search for PluginSimple usage. Expect: At least one occurrence.
rg --type typescript 'PluginSimple' src/main/webapp/app/shared/pipes/html-for-markdown.pipe.ts

22-22: LGTM: Method signature updated correctly. Verify impact on codebase.

The transform method signature has been correctly updated to use PluginSimple[] instead of ShowdownExtension[], which is consistent with the upgrade to markdown-it. This change follows the project's coding guidelines and aligns with the PR objectives.

To ensure this change doesn't introduce any issues, let's verify its impact on the rest of the codebase:

Please review the results of these checks to ensure that all related code has been updated accordingly.

src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-task.extension.ts (2)

3-3: LGTM: Import statement added correctly.

The new import for ArtemisTextReplacementExtension is correctly added and follows the Angular style guide for imports.


20-20: LGTM: Class extension updated correctly. Verify impact on usage.

The class definition has been updated to extend ArtemisTextReplacementExtension, which aligns with the new import. The class name follows PascalCase as required.

Please verify that this change doesn't break any existing functionality. Run the following script to check for any references to the old implementation:

Verification successful

Verified: No remaining references to ArtemisShowdownExtensionWrapper found. The getExtension() method is properly defined and utilized with the new class extension.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for references to ArtemisShowdownExtensionWrapper
rg --type typescript "ArtemisShowdownExtensionWrapper"

# Search for uses of getExtension method which was removed
rg --type typescript "getExtension\\(\\)"

Length of output: 166


Script:

#!/bin/bash
# Search for references to ArtemisShowdownExtensionWrapper in .ts and .tsx files
rg "ArtemisShowdownExtensionWrapper" -g "*.ts" -g "*.tsx"

# Search for uses of getExtension method which was removed in .ts and .tsx files
rg "getExtension\(\)" -g "*.ts" -g "*.tsx"

Length of output: 485

src/main/webapp/app/shared/markdown.service.ts (2)

28-28: Verify CSS class functionality after removing addCSSClass utility.

The htmlForMarkdown function call has been correctly updated to align with the new markdown-it library usage. The removal of the addCSSClass utility simplifies the code.

Please verify if the CSS class functionality previously provided by addCSSClass is still needed. If it is, ensure that it's handled appropriately in the new implementation. Here's a script to help verify the usage of CSS classes in markdown conversion:

#!/bin/bash
# Description: Check for CSS class usage in markdown conversion

# Test 1: Search for addCSSClass usage in the codebase
echo "Checking for any remaining addCSSClass usage:"
rg --type typescript "addCSSClass" src/main/webapp

# Test 2: Search for CSS class addition in markdown conversion util
echo "Checking for CSS class addition in markdown conversion util:"
rg --type typescript "class.*=.*" src/main/webapp/app/shared/util/markdown.conversion.util.ts

# Test 3: Search for markdown-it plugins that might handle CSS classes
echo "Checking for markdown-it plugins that might handle CSS classes:"
rg --type typescript "markdown-it-.*class" src/main/webapp

If the CSS class functionality is still needed but not handled by the new implementation, consider adding a markdown-it plugin or custom renderer to handle it.


3-4: LGTM: Import statements updated correctly.

The import statements have been updated to reflect the transition from showdown to markdown-it. The htmlForMarkdown import path change and the addition of PluginSimple type from markdown-it are appropriate.

To ensure consistency across the codebase, let's verify the usage of these imports:

Verification successful

Verified: Import statements updated correctly.

All showdown imports have been removed, and the new imports for htmlForMarkdown and PluginSimple from markdown-it are properly integrated across the codebase.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of new imports across the codebase

# Test 1: Check for any remaining showdown imports
echo "Checking for any remaining showdown imports:"
rg --type typescript "from 'showdown'" src/main/webapp

# Test 2: Verify the usage of htmlForMarkdown from the new path
echo "Verifying usage of htmlForMarkdown from the new path:"
rg --type typescript "import.*htmlForMarkdown.*from 'app/shared/util/markdown.conversion.util'" src/main/webapp

# Test 3: Check for proper usage of PluginSimple type
echo "Checking for proper usage of PluginSimple type:"
rg --type typescript "import.*PluginSimple.*from 'markdown-it'" src/main/webapp

Length of output: 681


Script:

#!/bin/bash
# Description: Verify the usage of new imports across the codebase using file extensions

# Test 1: Check for any remaining showdown imports in .ts and .tsx files
echo "Checking for any remaining showdown imports in .ts and .tsx files:"
rg "from 'showdown'" --glob '*.ts' --glob '*.tsx' src/main/webapp

# Test 2: Verify the usage of htmlForMarkdown from the new path in .ts and .tsx files
echo "Verifying usage of htmlForMarkdown from the new path in .ts and .tsx files:"
rg "import.*htmlForMarkdown.*from 'app/shared/util/markdown.conversion.util'" --glob '*.ts' --glob '*.tsx' src/main/webapp

# Test 3: Check for proper usage of PluginSimple type in .ts and .tsx files
echo "Checking for proper usage of PluginSimple type in .ts and .tsx files:"
rg "import.*PluginSimple.*from 'markdown-it'" --glob '*.ts' --glob '*.tsx' src/main/webapp

Length of output: 2857

src/test/javascript/spec/component/lecture-unit/text-unit/text-unit.component.spec.ts (1)

30-30: LGTM, but verify markdown output and id attribute removal.

The change from <h3> to <h1> aligns with the PR objective of upgrading the markdown library. However, please ensure the following:

  1. Verify that the new markdown-it library actually converts # Sample Markdown to <h1> without an id attribute. If it does generate an id, update the test expectation accordingly.
  2. Check if the removal of the id attribute is intentional and doesn't affect any functionality or accessibility features in the actual component.

To confirm the markdown-it output, you can run the following script:

This will show the actual HTML output from markdown-it for the given markdown input.

package.json (6)

44-44: LGTM: Addition of @vscode/markdown-it-katex

The addition of @vscode/markdown-it-katex (version 1.1.0) aligns with the transition from showdown to markdown-it for markdown processing. This plugin replaces the functionality previously provided by showdown-katex.

Note: As mentioned in the PR objectives, there might be a future transition to a different library for KaTeX support. Keep this in mind for future updates.


61-63: LGTM: Addition of markdown-it and related plugins

The addition of markdown-it (v14.1.0) and its plugins (markdown-it-class v1.0.0, markdown-it-highlightjs v4.2.0) directly addresses the PR objective of replacing showdown with markdown-it. These up-to-date versions provide the core functionality for markdown processing and maintain features like syntax highlighting.

This change aligns perfectly with the goal of upgrading the markdown library as stated in the PR objectives.


78-78: LGTM: Addition of turndown library

The addition of turndown (v7.2.0) is noted. This library allows for conversion of HTML to Markdown, which could be useful in maintaining compatibility or adding new functionality.

Could you please clarify the specific use case for turndown in the project? This addition wasn't explicitly mentioned in the PR objectives, so understanding its role would be helpful for future maintenance.


132-132: LGTM: Addition of type definitions for markdown-it and turndown

The addition of type definitions for markdown-it (v14.1.2) and turndown (v5.0.5) is a good practice. These definitions will help maintain type safety and improve developer experience when working with these libraries in a TypeScript environment.

Also applies to: 137-137


Line range hint 1-1: LGTM: Removal of showdown-related dependencies

While not visible in the provided diff, the removal of showdown-related dependencies (showdown, showdown-highlight, showdown-katex, and @types/showdown) is a crucial part of this PR. This change directly addresses the security vulnerability mentioned in issue #9333 and aligns with the main objective of replacing showdown with markdown-it.

This removal is a significant step in improving the security of the project by eliminating the vulnerable showdown library.


Line range hint 1-193: Overall assessment: Successful transition to markdown-it

The changes in this package.json file successfully implement the transition from showdown to markdown-it for markdown processing. Key points:

  1. All necessary markdown-it related dependencies and plugins have been added.
  2. Corresponding type definitions have been included for improved TypeScript support.
  3. The addition of turndown provides HTML to Markdown conversion capabilities.
  4. Showdown-related dependencies have been removed, addressing the security vulnerability from issue showdown-2.1.0.tgz: 1 vulnerabilities (highest severity is: 5.3) #9333.

These changes align well with the PR objectives and should result in a more secure and maintainable markdown processing system for the project.

src/test/javascript/spec/helpers/sample/problemStatement.json (1)

Line range hint 1-15: Overall assessment: Improvements align with PR objectives

The changes in this file successfully enhance the visual representation of test statuses, which aligns well with the PR objectives of upgrading the markdown library and improving the UI. The addition of SVG icons provides clear and scalable visual indicators.

While the implementation is solid, consider the suggested accessibility improvements to make the UI more inclusive for users relying on screen readers.

src/main/webapp/app/exercises/quiz/shared/short-answer-question-util.service.ts (1)

376-376: LGTM: Improved code readability

The addition of a blank line before the firstWordIndex calculation improves the code's readability by visually separating the variable declarations from the subsequent logic. This change aligns with good coding practices for maintaining clean and easily readable code.

src/main/webapp/app/exercises/programming/shared/instructions-render/programming-exercise-instruction.component.ts (2)

19-19: LGTM: Import change aligns with library upgrade

The import of PluginSimple from 'markdown-it' is consistent with the PR objective of upgrading from Showdown to markdown-it library. This change appropriately reflects the transition in the markdown processing library being used.


83-83: LGTM: Property type updated for markdown-it compatibility

The markdownExtensions property type has been correctly updated from ShowdownExtension[] to PluginSimple[]. This change ensures type safety and compatibility with the new markdown-it library's plugin system.

src/main/webapp/app/exercises/programming/shared/instructions-render/extensions/programming-exercise-plant-uml.extension.ts (3)

3-4: Appropriate imports for extended functionality

The added imports for ArtemisTextReplacementExtension and escapeStringForUseInRegex are necessary for extending the base class and utilizing utility functions.


16-16: Class correctly extends ArtemisTextReplacementExtension

Changing the class to extend ArtemisTextReplacementExtension aligns with the updated architecture using markdown-it and promotes better code reuse and maintainability.


27-29: Proper initialization of the superclass

The call to super(); in the constructor ensures that the superclass is properly initialized, which is essential when extending a base class.

Copy link
Contributor

@JohannesStoehr JohannesStoehr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on ts2 and looked through the code. Looks good to me

Copy link
Contributor

@florian-glombik florian-glombik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approve after merging develop

Copy link
Contributor

@pzdr7 pzdr7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good. Works well on TS2 👍

Copy link

⚠️ Unable to deploy to test servers ⚠️

Testserver "artemis-test2.artemis.cit.tum.de" is already in use by PR #9595.

@github-actions github-actions bot added the deployment-error Added by deployment workflows if an error occured label Oct 27, 2024
@HawKhiem HawKhiem added deploy:artemis-test2 and removed deployment-error Added by deployment workflows if an error occured labels Oct 27, 2024
@HawKhiem HawKhiem temporarily deployed to artemis-test2.artemis.cit.tum.de October 27, 2024 19:09 — with GitHub Actions Inactive
@krusche krusche merged commit 432efc3 into develop Oct 27, 2024
92 of 101 checks passed
@krusche krusche deleted the chore/markdown-it-replacement branch October 27, 2024 19:27
@krusche krusche added this to the 7.6.4 milestone Oct 27, 2024
Copy link

@HawKhiem HawKhiem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on TS2. Approve after the merge (Everything works as described)

Screenshot 2024-10-27 202124
Screenshot 2024-10-27 202151
Screenshot 2024-10-27 202211

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client Pull requests that update TypeScript code. (Added Automatically!) documentation ready for review tests
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

showdown-2.1.0.tgz: 1 vulnerabilities (highest severity is: 5.3)