COBOL textmate - the begin/end rules no longer work since the management of the line number and identification zones was added

[FALLAI-Denis]

Development environment used

• Z Open Editor version: 4.1.0
• Editor Platform
  • Visual Studio Code
  • Red Hat CodeReady Workspaces
  • Eclipse Che
  • Standalone Theia
• Editor Platform Version: 1.88.0
• Operating System: Windows 10
• Java Version: 1.17
• Related to RSE API?
  • RSE API Plugin version:
  • Zowe CLI version:
  • Node.js version:
• Logs attached:no

Problem Description

The begin/end rules no longer work since the management of the line number and identification zones was added.

This is due to the addition of the following rule at the start of the textmate grammar:

    {
      "match": "(^.{6})(.{0,66})(.*$)",
      "captures": {
        "1": {
          "name": "entity.name.function.seqnum"
        },
        "2": {
          "patterns": [
            {
              "include": "source.cobol"
            }
          ]
        },
        "3": {
          "name": "entity.name.function.seqnum"
        }
      }
    },

It has the effect of isolating the text in columns 7 to 72 and it is on this isolated text that the other rules are applied, therefore the rules based on begin / end (or while / end).

If this rule is disabled, then begin/end based rules work.
This impacts the management of multi-line EXEC xxx/END-EXEC sequences.

Observed behavior

Expected behavior

With additional management of line number and identification zones.

[FALLAI-Denis]

Related issue:

• COBOL textmate - Support for comment in SQL statements #401

[FALLAI-Denis]

Hi,

I think that to resolve this problem we need to review the organization of the rules in the cobol.json file:

1. first process / isolate lines blocks, and potentially nested blocks,
2. in each block process each line, for captured lines and begin / end lines: it is at this stage that the line must be splitted between columns 1-6, 7, 8-72, 73-80, beyond 80,
3. process each token of 8-72 part of splitted line.

Blocks are:

• "exec xxx" / "end-exec": call to external textmate with a local filtering for COBOL lines or parts of line (splitting line, COBOL comments, COBOL variable, ...)
• comment paragraphs in Identification Division (ie: Remarks paragraph wich is a comments lines block)
• others ?

Implementing "Division" blocks give capacity to implements fine grain rules:

• level number for data-item in Data Division: they are level before to be number
• pure numeric paragraph names in Procedure Division: they are identifier before to be numeric

Blocks can be:

• "begin" / "end" when an "end" token exists
• "begin" / "while" (not) when no end token exists (the "end" is the start of another block)
• a single line (last rule in blocks rules)

Also consider "bad formed" blocks: missing end pattern in source text, so perhaps "begin" / "while" should be privilegied.

This means declaring a repository of named rules and referencing these named rules according to the sequence indicated above.

I am currently working on such an implementation and will come back here to share my work.

Difficulty is to isolate active tokens in begin / end / while rules (deal with commented tokens): trying to implement lookhead / lookbehind regex for this.

[sdaimwood]

Hi,

I think that to resolve this problem we need to review the organization of the rules in the cobol.json file:

1. first process / isolate lines blocks, and potentially nested blocks,
2. in each block process each line, for captured lines and begin / end lines: it is at this stage that the line must be splitted between columns 1-6, 7, 8-72, 73-80, beyond 80,
3. process each token of 8-72 part of splitted line.

Blocks are:

• "exec xxx" / "end-exec": call to external textmate with a local filtering for COBOL lines or parts of line (splitting line, COBOL comments, COBOL variable, ...)
• comment paragraphs in Identification Division (ie: Remarks paragraph wich is a comments lines block)
• others ?

Implementing "Division" blocks give capacity to implements fine grain rules:

• level number for data-item in Data Division: they are level before to be number
• pure numeric paragraph names in Procedure Division: they are identifier before to be numeric

Blocks can be:

• "begin" / "end" when an "end" token exists
• "begin" / "while" (not) when no end token exists (the "end" is the start of another block)
• a single line (last rule in blocks rules)

Also consider "bad formed" blocks: missing end pattern in source text, so perhaps "begin" / "while" should be privilegied.

This means declaring a repository of named rules and referencing these named rules according to the sequence indicated above.

I am currently working on such an implementation and will come back here to share my work.

Difficulty is to isolate active tokens in begin / end / while rules (deal with commented tokens): trying to implement lookhead / lookbehind regex for this.

Hi @FALLAI-Denis, thank you for investigating this. Please do keep us updated on your findings. Will mark this and #401 as enhancements for now and investigate what we can do as well.