GRML Specification

Overview

This page describes the Grbl_Esp32 version 2 text-based configuration mechanism is detail. There are simpler descriptions that are probably enough to get you started...

Grbl_Esp32 is configured (adapted to a particular machine) at run-time by reading a config file from the local file system. The config file is a text file that contains a hierarchical description of which machine features are present, the assignment of controller pins to those features, and other parameters that affect the machine behavior.

The name of the config file is given by the setting $Config/Filename. Its default value is "config.grml", but you can change it to another name.

If an error occurs when processing the config file, Grbl_Esp32 will fall back to a rudimentary "safe" configuration so you can interact with the system to fix problems.

Uploading a Config File

You can upload a config file over WiFi by using the file upload button in the ESP3D tab. If you upload a file named "config.grml", Grbl_Esp32 will use it with no additional setup required. If you want to use a different name, you will have to change the $Config/Filename setting to the new name.

If you are using PlatformIO to develop Grbl_Esp32, another way to upload a config file is by editing the existing file "Grbl_Esp32/data/config.grml" then running the PlatformIO command pio run -t uploadfs.

What is GRML

GRML (Grbl Markup Language) is a configuration file format that is inspired by YAML (Yet Another Markup Language). GRML is less powerful than YAML, but also much simpler, so easier to parse with fewer corner cases to consider. GRML is close enough to YAML that many YAML syntax-highlighting tools work reasonably well with GRML.

You can create and modify GRML files with an ordinary text editor. Many "programmer-grade" editors have YAML syntax support with indentation assistance, pretty-printing, and automatic conversion of tabs to spaces (tabs are forbidden in YAML and GRML files). GRML is similar enough to YAML that "YAML mode" typically works for editing GRML files.

A GRML document looks like this:

name: "XY Laser"
board: "ESP32 Dev Controller V4"

# This comment line is ignored

step_type: rmt

axes:
  number_axis: 3

  x:
    gang0:
      stepstick:
        direction: gpio.14:low
        step: gpio.12

  y:
    gang0:
      stepstick:
        direction: gpio.16:low
        step: gpio.5

laser_mode: true

comms:
  wifi_sta:
    ssid: "myWifi Access Point"

Each non-empty GRML line is either an ignored comment (if the first non-blank character is '#'), a section (if there are no non-blank characters after the ':') or an item (if there are non-blank characters after the colon). Blanks must be true space characters; tabs are not permitted.

In sections and items, the part before the ':' is called the "key". In items, the part after the ':' is called the "value".

A GRML document represents a hierarchical tree structure where indentation determines the level within the hierarchy. Sections and items with zero indent are at the top level of the tree. Each section introduces a new node within the tree; the section's children are indented more deeply. In the example above, the top level contains "name", "step_type", "axes", "laser_mode", and "comms". "axes" is a section whose direct children - indented by 2 spaces - are "number_axis" and "x". "x", in turn, introduces another level whose only direct child is "gang0", and so forth.

The indentation need not be in multiples of 2. The first child of any section determines the indentation for the direct children of that section. All other direct children must be indented the same as that first one. The indentation is determined on a section-by-section basis, so it is okay for each section to have a different starting indentation - but please be consistent for human readability. We recommend an indentation pitch of 2 spaces.

Keys - the part of sections and items preceding the ':' - may not contain blanks. Values - the part of items after the ':' may contain blanks, but only if the first non-blank character is either ''' or '"'. Thus, if we write:

my_key: this is a value

the value is just "this", and everything after is ignored. Conversely, in

my_key: "this is a value"  junk junk

the value is "this is a value", and again, everything after it is ignored.

At the syntactic level, every value is a string. That string might be interpreted to mean something else, such a number or true/false value, but the interpretation is done later, not in the code that scans the GRML document to map it to a tree.

GRML vs YAML Rationale

(You can skip this section if you just want to learn how things work, instead of engaging in "language lawyer" arguments.)

YAML is somewhat standard, and considerably more powerful than GRML. The power comes at a price. YAML supports things like typed data, arrays with different syntaxes for specifying them, references to other sections, and whatnot. Those features result in a complex parser as well as some ambiguous situations like "should 01235 be considered a integer or a string?". (Obviously it is an integer, but there are uses where you might want it to be a string.) In GRML, every value is a string whose interpretation is deferred to the object that needs to use it; that object knows what it wants.

The GRML restrictions result in a very simple - thus small and robust - parser, with enough expressive power for the task at hand. The lack of advanced YAML features makes the format easier to understand. The rules are very simple and there is very little syntax.

How is GRML Used

Grbl_ESP32 uses a GRML document to configure itself for a particular machine. Classic (AVR) GRBL is configured with combination of recompilation and $NNN (where NNN is a number) settings. Recompilation is necessary for things like changing pin assignments and enabling or disabling optional features, while $NNN settings could change, without recompiling, a selected set of tuning parameters like motor speeds. Grbl_ESP32 extended the $NNN settings with named settings like $Spindle/Type. The named settings enabled a wide range of new features, but were not suitable for a complete machine setup including pin assignments.

Now, with GRML-based configuration, it is possible to configure Grbl_ESP32 without recompiling the firmware. The GRML file describes (nearly) everything that used to require recompilation to effect a change. The exception is "custom code" - if your machine is so different that it was necessary to add custom C++ code to handle its special features, you will still need to recompile to inject your custom code.

Most of the "machine.h" files that configured the firmware via "#define" statements are now replaced by "config.grml" text files that configure a single precompiled binary version of Grbl_ESP32 at runtime.

How it Works

The GRML config file tells GRBL which settings to use for the various supported features. This includes not only the features that could be configured via "$name=value" commands, but also the fundamental machine setup of assigning I/O pins to functions, selecting and configuring motor and motor driver types, axis ganging (multiple motors per axis) and choosing between different endstop arrangements.

This is a tricky task when you consider the problems of setting up different ganging arrangements for different axes, while also assigning endstops and motor drivers to axes and gangs, and assigning pins of various types (GPIO, I2SO, I2C) to functions. Doing so with a static collection of named settings would have required an unmanageable number of settings.

Instead, when Grbl_ESP32 starts, it reads a "config.grml" file that tells it how to create a configuration tree structure on the fly. The tree maps to a set of C++ objects (in the object-oriented programming sense) that perform various subtasks like stepping a motor (or a ganged set of motors), sensing a probe or limit pin, changing the speed of a spindle, etc.

This heavy use of object-oriented programming techniques in conjunction with a dynamically created tree structure results in a system of extraordinary capability and flexibility that can be adapted to a wide range of different machines without recompilation.

GRML and Objects

The Grbl_Esp32 code is organized around a set of C++ objects that perform various tasks. Those objects fall into various categories - "abstract classes" in C++ parlance - that perform specific tasks. For example, the Motor "abstract class" knows how to manage the "step" and "direction" signals of a motor, or for "smart motors", how to tell it to move to a given location. There are specific "specialized classes" for particular motor drivers like ordinary stepsticks, Trinamic stepper drivers of various flavors, unipolar motor drivers, servos, etc. The top level code for motion control does not care which motor type is in use; it simply issues a "step" request to the motor object that is assigned to a given axis and lets the object decide how to do it.

Similarly, there is an abstract class for spindles which responds to speed and direction requests. There are specialized classes for different spindle types like PWM-controlled spindles and lasers, DAC-controlled spindles, and VFD of various flavors. Again, the top-level code does not care which is being used; it just sends a "set_rpm" request and lets the Spindle object sort out what to do.

Each object has a set of configuration items that can differ from object to object. For example, a VFD spindle might need to know the minimum and maximum RPM plus the UART pins necessary for serial communication to the VFD, while a PWM spindle might need minimum and maximum RPM plus things like the PWM frequency.

In addition, the item values for objects of the same type can differ from axis to axis. For example, a motor on the X axis might use 200 steps per mm, while a similar motor on the Y axis might use 100 steps per mm.

The GRML file format supports all of those requirements. It lets you assign motors of various types to axes and lets you set configuration items according to the particular type of object that is in use, on a per-instance basis.

Canonical GRML Config File

A complete GRML config file describes the entire hierarchy of things that can be configured in GRBL as shown:

<top level items>

axes:
  <items that are common to all axes>
  x:
    gang0:
      endstops:
         <endstop items>
      <motor type>:
         <items specific to that motor type>
    gang1:
      endstops:
         <endstop items>
      <motor type>:
         <items specific to that motor type>

  y:
    <subordinate nodes just like for x>

  <additional axes z, a, b, c similar to x>

comms:
  wifi_sta:
    <wifi station items>
  wifi_ap:
    <wifi access point items>

i2so:
  <i2so items - pin assignments>

spi:
  <spi items - pin assignments>

user_outputs:
  analog0: <pin>
  ...
  analog3: <pin>
  digital0: <pin>
  ...
  digital3: <pin>

coolant:
  mist: <pin>
  flood: <pin>

control:
  safety_door: <pin>
  <other control pins: reset, feed_hold, cycle_start, macro0 .. macro3>

probe:
  pin: <pin>

sdcard:
  <sdcard items - pin assignments>

<spindle_type>:
  <items specific to that spindle type>

The structure of the canonical file mirrors the capabilities of Grbl_Esp32 and GCode. For example, the axes section reflects motion control with motors and limit switches, while the coolant, control, user_outputs and probe sections assign pins to the corresponding "switched" GCode functions.

A GRML file need not mention everything in the canonical tree, but everything that is mentioned must correspond to the structure shown. The order of items within a section is unimportant, but the hierarchy is critical. You cannot put "analog0" inside a "probe" section. "endstops" must be directly underneath a "gangN" section, not up a level underneath "x".

Anything that the GRML file omits will be defaulted. The most common default is "nothing happens when you invoke that feature". An example is the control pins; any control pin that is not specified in GRML will never signal the corresponding event. Similarly, if you do not assign a motor to (a gang underneath) an axis, GCode moves on that axis do not cause physical motion.

A few defaults result in active features. For example, the defaults for spi and sdcard use the ESP32 pins that are commonly assigned to those functions. wifi_ap defaults to SSID "GRBL_ESP" with the IP address 10.0.0.1 so you can connect to an unconfigured machine via WiFi to upload a configuration file.

An empty GRML file is equivalent to the "test drive" machine definition in the old scheme; you can interact with GRBL, but there will be no motion.

Generic Sections

Consider this GRML config file fragment:

axes:
  x:
    gang0:
      stepstick:
        direction: gpio.14:low:pu

The "axes", "x", and "gang0" sections are rather ordinary, with unchanging names in the canonical tree, but "stepstick" has special considerations. "stepstick" is an example of a "motor type". There are numerous motor types, including "null_motor", "standard_stepper", "stepstick", "tmc_2130" (and several other Trinamic variants), "unipolar", "dynamixel2", and "rc_servo". New motor types are likely to added over time.

Different motor types have different sets of configuration items. Motor types that are stepper motor variants will typically have "step" and "direction" items to define the associated pins, and, depending on the specific variant, might have additional items for things like disable pins, mode select pins, and run and hold currents for Trinamic drivers. Other motor types might have totally different configuration item sets, such as phase pins for Unipolar motors and serial communication pins for "smart" motors like Dynamixels.

Sections like this, where there is a general capability like "motor" for which you must choose a specific type are called "generic sections".

In the canonical GRML file, generic sections are shown with a generic secion name like "<motor_type>:" instead of a literal name. The generic name "<motor_type>" must be replaced by the name of a specific motor in a specific config.GRML file.

In addition to motors, generic sections are used for spindles. There are many Spindle types, such as "PWM", "Laser", "10V", "BESC", "Relay", and various kinds of VFDs like "Huanyang", "YL620", and "H2A". As with motors, the configuration items for spindles differ for different spindle types. The canonical GRML file shows the generic spindle subsection as <spindle_type>, which must be replaced by a specific spindle type name.

Pin Syntax

Many configuration items assign an I/O pin to some function. Pins are specifed by this format:

  <pin_function> : <pin_type>.<pin_number>:<attribute>:<attribute>...

as in this example which sets the X axis limit switch to GPIO 16, active low, with internal pullup enabled:

axes:
  x:
    gang0:
      endstops:
        dual: gpio.16:low:pu

<pin_function> is the GRBL functional behavior to which the pin is assigned. <pin_type> is either "gpio" or "i2so" (other types like "i2c" will probably be added soon). <pin_number> is a small number that is interpreted according to <pin_type>. For "gpio", <pin_number> is just the GPIO number. For "i2so", <pin_number> is the number of I2SO output in the serial chain. "gpio.5" and "i2so.5" are different pins despite the fact that both use the number 5.

":..." is an optional list of modifiers to apply to the pin. The attributes that are supported depend on pin_type, but there are some attributes that apply equally to all types. ":low" and ":high" can be used on all pins. ":high" - which is the default value if neither ":low" nor ":high" is mentioned - means the pin is "active high"; if the code writes 1 to the pin, it will go to the electrical "high" state, and on input, electrical "high" makes the pin read as 1. :low" means that the pin is "active low" - if the code writes 1 to the pin, it will go to the electrical low state, and electrical low on an input pin reads as 1.

This method of controlling a pin's active level replaces the "inversion mask" settings in classic Grbl. Those settings were confusing, requiring the user to perform bitwise arithmetic or look up values in a table, and difficult to extend to situations like ganged motors.

GPIO pins can further be modified with ":pu", indicating that the ESP32's internal pullup should be turned on, and ":pd" to turn on the internal pulldown. Note that this setting does not tell the code about the the presence or absence of external pullup/down resistors. It simply controls whether or not to enable the internal pull resistors. If you have a sensor or switch setup that needs a pullup, it is up to you to either add one externally or to engage the internal one by saying ":pu" in the pin specification. (Since ESP32 internal pullups are rather weak, they are useful for signals that do not go off-board; it is better to use stronger external pullup resistors for off-board signals. You could, however, use an external pullup and also specify ":pu" to prevent false triggering when the external sensor is disconnected.)

":pu" and ":pd" do not apply to I2SO signals, since they are output-only and do not have software controlled pulling resistors. Attempts to apply impossible attributes will report an error.

There is no need, and no provision, for setting a pin's direction to "input" or "output". That knowledge is implicit in the function to which the pin is assigned. For example, motor direction signals must be outputs, so the GRBL startup code applies that knowledge automatically when it sets up the pin. Similarly, a pin assigned to probe must be an input. When the Grbl startup code processes the pin assignments, it notices impossible assignments, reporting them as errors. For example, I2SO pins are output-only, so if you try to use one as a limit pin, you will get an error.

The default value for pins is active high, no pullup or pulldowns.

How the GRML/Tree Creation Code Works

The Grbl_ESP32 contains a set of C++ objects that closely mirror the canonical tree shown above. Tree sections correspond to C++ classes, while items correspond to member variables within those classes. The startup code processes the GRML configuration file to create an internal tree of C++ objects that mirrors the configuration file.

Each such C++ class contains a "key list" of the items and sections that it understands. For items (which have a value after the ':'), the list entry has the key name (corresponding to the part of the GRML line before the :), a variable to hold the associated value, and the data type of that value. For sections (with no value after the ':'), the list entry has the key name, a variable to point to a lower level in the tree, and the C++ class type for that lower level.

Here's how it works, step by step:

Tokenizing

Each valid, non-blank, non-comment GRML line has

• A "key", which is the first sequence of non-space characters before the ':'
• An "indent", which is the number of space characters before the key
• An optional value, which is the string following the ':'

Those three things are collectively known as a "token".

Processing a Section

Each section within the GRML file is handled as follows:

• Process each token whose indent is the same as the section indent by searching for the token's key in the section's key list.

** If there no match, report an unmatched key and proceed

** If there is an item match, convert the value string to the indicated data type and store it in the variable.

** If there is a section match, create a new object of the indicated class, store its pointer in the variable and recursively process a new (sub)section with a greater indent, using the new object's key list.

• Tokens with a greater indent than the current section indent, but not part of a subsection, are skipped with an error message.

• A token with a lesser indent than the current section indent indicates that the section is complete and processing of the enclosing section resumes.

Top Level Processing

• Open a file named "config.grml" on the local (e.g. SPIFFS) filesystem. If that fails, use a built-in default file.

• Create a top-level object of the MachineConfig class, set the section indent to 0, set the current section to that object, and process that section as above. Due to the recursive processing of subsections, this will ultimately result in the entire file being processed.

Fixup and Validation

After the GRML file has been handled, the resulting configuration is checked in two phases. The first "afterParse" phase traverses the in-memory configuration tree and fixes problems that can be easily corrected. The typical use of this fixup is to create a default setup for required sections that are not mentioned in the GRML file. This makes it possible to have an empty or nearly empty GRML config file, and still have an intact canonical tree. After the afterParse phase, the in-memory configuration tree is complete so Grbl can access every required part of it, even if some of the features might "do nothing".

The second "validate" phase traverses the in-memory tree again. looking for invalid or conflicting conditions. The most common invalid condition is when the GRML file specifies a section but does not specify one of the pins that it requires. There are some sections that can use default pins, such as in the spibus section which will use the "standard" ESP32 SPIbus pins. More commonly, there are no reasonable default pins, so you must specify the pins. The validate phase can also check for items with invalid numerical values, sets of items with mutually-conflicting values, or any other situation that does not make sense.

If either the afterParse phase or the validate phase fails, the system will issue a "Validation Error" message describing the problem and ??? DO WHAT ???. Typically, since afterParse's job is to clean up things that it can easily fix, it will not "fail", but it might report the fixup steps that it undertook. Problems that are discovered by validate generally indicate that the machine will not work.

Making a New Configuration Item

To create a configuration item, you need to decide which section (at the source code level, which C++ class) it belongs in. Top-level items go in the MachineConfig class, items that apply equally go all axes go in Axes, items that are specific to one axis (i.e. that can have a different value for each axis) go in Axis, etc.

As an example, let's say that you want to add a true/false item called "oily" to the "coolant" section.

In CoolantControl.h, after the line "public:", add this line:

    bool _oily = false;

That creates a variable that you could later access from external code with config->_coolant->_oily. Within the CoolantControl class code, you could access it with just _oily.

To make the variable configurable via GRML, edit the "group" function in CoolantControl.cpp to add the line:

    handler.item("oily", _oily);

Then in the GRML file, you could say:

coolant:
    oily: true

The general procedure is: Add a variable of the desired data type in the enclosing section's class declaration. In that class's group() method, add handler.item("KEYNAME", VARIABLE_NAME);

Making a New Configuration Section

Configuration sections are created similarly to configuration items with one large difference and one small difference. The large difference is that you have to make a new C++ class for the section. That is a big topic that is beyond the scope of this discussion. The additional things needed to make the new class work within the GRML configuration framework are not too complicated:

• The new class must inherit from Configuration::Configurable, e.g.

class MyClass : public Configuration::Configurable {

• The new class needs a "group()" method to serve as a key list for the section, e.g.

public:
    void group(Configuration::HandlerBase& Handler) override {
        handler.item("my_key1", _myvariable1);
        handler.item("my_key2", _myvariable2);
    }

where _myvariabl1e and _myvariable2 are member variables of the new class that can be configured via GRML.

• The new class can optionally have "validate()" and "afterParse()" methods to check the final configuration of the class and to perform cleanup or defaulting actions that need to be done after the GRML file handling is complete. If those methods are not present, they default to no-ops.

• To place the new class within the canonical tree, you must choose the existing class that will be its parent, and

** In the parent class's list of member variables, add a class pointer variable for the new class, e.g.

<inside parent class declaration>
public:
    MyClass* _myclass = nullptr;

** In the parent class's "group()" method, add this:

<inside parent class group() method>
   handler.section("my_section_name", _myclass);