Qt, OpenGL, C++ Parsing

[infnorm]

I've been experimenting with some relatively major changes to bCNC.
I have loved the functionality in bCNC, the combination of CAM, the ability to run on modest RPI hardware, and its reliability. However, I wanted to improve the performance, visualization, etc.

C++ Parsing

By rewriting CNC and GCode classes mostly in C++, I was able to get a significnant speedup
9.5MB gcode takes 7.9s to read/generation motion paths for with Python code and only 0.22s to read in C++ for a 35x speedup. A key difficulty here is supporting complicated expressions. My solution here is to observe that most of the time gcode has no expressions, so just make that fast and in C++. If there is a construct that uses python expressions, just call back into python to do those lines. This seems like a good solution, because you pay for what you use. Python expression laced gcode will never be super fast, but hoepfully there will only be some of it.

A major point with this change is that the way the data is handled has to be made much more efficient. Blocks consist of strings, until you open them in the editor and then they are split into newlines. the C++ gcode simulation spits out a numpy array of a point per row which can be drawn as line segments. The name of the game is minimizing the amount of looping in Python. Even with a 10MB file, the GUI stays responsive when you don't try to edit everything at once.

OpenGL visualization

I wrote a OpenGL visualizer that allows selection, quick drawing, tumbling of camera. This works in Raspberry Pi 3b and above, Linux, Windows and MacOS. It is in particular very difficult to get this working on Raspberry PI, because of its ancient OpenGL implementation, so I carefully avoided newer features. This precludes the use of things like ModernGL though.

Rewritten UI in Qt

Qt is much faster than Tk and pretty well supported (tcl/tk is one of the slowest scripting languages because it converts every value to a string and back). It's available on every platform (I targeted PySide2 because that was available on RPI). Furthermore, it allows things like dockable widgets, dark/light themes, etc. This is potentially a lot of work. It may be possible to make bCNC support multiple GUI frontends, but I don't know that this is a great idea. My thought is to centralized commands and events in a gui agnostic way to enable this, but that also makes things work much better in general.

I considered several options here

• Rewrite gui in html5/css/js - Web browsers are the most modern UI toolkits there are. It would look the best. Performance of web browsers on RPI <4 seems not sufficient to me, so I avoided this. Also, if this is the route people prefer then probably cncjs is a better choice
• Try to put GL into Tkinter windows. There were threads on this in the bCNC issues, but it seemed like this never came to a perfect place. Also, I found that the PyOpenGL bindings on RPI are not very reliable. They work for basic things, but I needed to use Qt's abstraction library to load values into shader programs.
• PyGtk - PyGtk doesn't perform well on non linux platforms like Mac OS. Plus, the opengl binding issue.
• Qt - this was the option I went with.
• Hybrid - use Tk for most of the UI and Qt for just OpenGl. This seemed like a minimal change, but one that was not super easy to do. Probably each of their message/pump/event loops would have to be in separate threads.

I think the Qt GUI behaves pretty well. It doesn't look particularly modern, but that is possible to tweak with Qt styles (i.e. get a flatter look. Right now it adheres to each platform's native look mostly.

What works what doesn't.

Works

• Basic software
  • Loading files
  • Editing and removing blocks
• CNC operation
  • Basic connection to grbl
  • Running CNC programs
  • Jogging setting WCS
• Visualization
  • OpenGL visualization
  • Camera selection

Doesn't work

• CAM
• Progress bar when running cam program not quite right.
• Highlighting visualization as cam blocks run not quite right (maybe it is as good as bCNC is generally at this... i was thinking of using line number grbl reporting to improve this)
• All variables in CNC.vars[] are not updated quite right yet.
• Fallback to expressions has a hook but it hasn't been finished
• Multiple selection
• Bounding box selection
• Probe page
• Z height depth map compensation
• Shortcuts, saving preferences, hotkeys, other UI niceties

I'm not sure the best way to handle this change, and I'm not yet ready to share the code, but I wanted start a discuss here at this time at least, before I went any further.

Demo videos

OpenGL canvas and running connected to GRBL
https://youtu.be/W-3MoK6CjaU

Demo of loading a 10MB file almost instantly
https://youtu.be/1qp6bpoK3oM

GCode editing
https://youtu.be/R1hdidhtygs

OpenGL canvas selection
https://youtu.be/y36fjpcXJTU

Screenshot

[LittlePierre]

Looks great

[MARIOBASZ]

That's so good!!

[rschell]

Looks interesting. Happy to help in whatever form.

I have both a RPI 4b and amd64 hardware to run bCNC on. These interface to a grbl 1.1g and to a 32 bit teensy GrblHAL controller.

[infnorm]

Sure, I'll send a ping out when I'm ready!

[hooperbloob]

I'd be happy to help with development also, it looks great.

[infnorm]

Not much updates, I've been looking into the world of material simulation to see how plausible it would be to simulate stock during simulation. It's a very interesting thing, that would make bCNC way more useful, but it does incur a lot of library dependencies maybe not worth it.

I'd also love to have some commentary on the OP from @Harvie

[LittlePierre]

Dear @infnorm

I had a look at PySide2 & PySide6 and integration with PyOpenGL.
I encountered some display issues due to bad support of wayland (Ubuntu) that disappear with Xorg display.

I have a few questions / remarks :

• Is your main app (the entry of the program) written in Python, with PySide2 ? I understand that yes, can you please confirm ? Do you integrate 3d with PyOpenGL ?
• Do you bind C++ objects with clang or with ctypes ? Or other ?
• FreeCad uses a different approach, the main app is written in C++ with Qt, and they bind python objects with "PyObject" from <Python.h>
• I had a look at libs used by CAM module. I think that both bpath.py and bmath.py should be re-written in C/C++ because they both make heavy calculations. Maybe some more libs.

Thanks for your works,
All the best
Pierre

[infnorm]

Thanks for the questions. I haven't tested on Wayland, but most platforms I am currently using still use Xorg. I wanted to PySide6, but it is not supported on Raspberry Pi. PyOpenGL doesn't seem to work at all on Raspberry Pi. Given what versions of OpenGL work on RaspberryPi, I had to be very careful about how I wrote my glsl shaders.

Is your main app (the entry of the program) written in Python, with PySide2 ? I understand that yes, can you please confirm ? Do you integrate 3d with PyOpenGL ?

I left the main app in Python. I integrated with 3d mainly using the Qt OpenGL bindings because PyOpenGL doesn't really work on RaspberryPi 3 which is a platform I want to support.

Do you bind C++ objects with clang or with ctypes ? Or other ?
I used pybind11. I communicate mainly with np array through the array interface. That makes that pretty easy. Bindings don't matter that much, I've done it many ways in the past ... swig, pybind, boost::python, etc. pybind seems fine.

FreeCad uses a different approach, the main app is written in C++ with Qt, and they bind python objects with "PyObject" from <Python.h>
Well pybind uses the Python API (PyObject*) in its implementation. At the end of day you are making a extern "C" {} C ABI compatible shared library which is all that Python can natively speak.

In terms of what should own the world, python or C++, keeping Python owning the world makes building much easier. The way I have it setup python setup.py can build on windows, linux, mac os with minimal effort. If you go the FreeCad approach, maintaining all the build system stuff is a major pain. bCNC is mostly a Python program, keeping the main() in Python seems in the spirit of that :).

I had a look at libs used by CAM module. I think that both bpath.py and bmath.py should be re-written in C/C++ because they both make heavy calculations. Maybe some more libs.

I rewrote most of major calculations in the cam which was mostly interpretering gcode. Some things like z height compensation I have not implemented yet, but will also be in C++.

[LittlePierre]

Thank you very much. This loooks great.
I would be happy to help if needed.
This is going to bring a touch of modernity to bCNC and it's old tk interface.

Thank you for your works.
Pierre

[infnorm]

Thanks... this has been a little delayed (har...) but I'm working on picking it up. My cnc machine build has progressed to the point where I'm ready to test with it some more, and then I'll probably do a first code drop.

[LittlePierre]

Some considerations

- libfxfrw
I have been playing with libdxfrw
It is written in C++ and seems to be much more complete and robust than bCNC/lib/dxf.py

- Path
Also, another thing is that in bCNC, Path objects are a list of Segments objects.
But a path can include Segments where end point does not match next segment start point.

In Freecad SW , they forbid this, because it generates errors when making offset.

- eq
in bCNC/lib/bpath.py

# -----------------------------------------------------------------------------
# Compare two Vectors if they are the same
# -----------------------------------------------------------------------------
def eq(A, B, acc=EPS):
    if A is None or B is None:
        return False
    d2 = (A[0] - B[0]) ** 2 + (A[1] - B[1]) ** 2
    err = (
        acc * acc * ((abs(A[0]) + abs(B[0])) ** 2
                     + (abs(A[1]) + abs(B[1])) ** 2 + 1.0)
    )
    return d2 < err

Why does it return d2<err ?
I think this generates errors in offset, and some other Path functions.
I would return d2<acc, no ?
Plenty of bpath.py functions use eq, and I think there's something wrong.

[rar8000]

This is great. I'd love to help and test.

[cmdb-web]

This great. I'd also like to help testing - running bCNC on RPi3B+ and OPi Plus (older H3 hw running with armbian bookworm)