Bidirectional compilation?

[jleonard-r7]

Hi,

Is it possible to reverse a Python source file (or AST) to hissp? This would be a killer feature (enabling emacs structural editing for emacs users and yet still seemingly appearing as plain Python to regular users).

[gilch]

Trivially, and not in the way you want. A string containing Python code is a valid Hissp program that compiles to that Python code. (In Lissp, you'd have to inject it with .# and escape certain characters. In readerless mode, it's just a normal Python string literal.) This wouldn't help you with structural editing at all.

For structural editing in Emacs you'd want to use Lissp's tuple language, but that only compiles to a subset of Python. Hissp's only special forms are lambda and quote, which compile to Python lambda expressions or Python data expressions (possibly including pickles). Non-special forms are calls (or method calls) or invocations of macros defined in terms of these. There's also some special preprocessing for certain atoms, but that's it. The compiled Python output has a close correspondence to the input Hissp, which you can pick up pretty easily by playing with the Lissp REPL, but given these restrictions, the results are completely unpythonic; it's not "plain Python to regular users".

Even if you were willing to implement a reverse transpiler for the tuple language, you'd need a way to express all of the Python language in terms of the Python subset. This is sort of possible, but would subtly change the meaning of the original program in numerous ways, and (of course) the Python you'd get back out of the Hissp compiler would look completely different from what you started with; it doesn't round trip. If you really need to represent all of the Python language programmatically, use Python AST, not Hissp's simplified subset. Unfortunately, Python AST is not an easy language to program in (and that's one of the reasons why Hissp exists).

You may instead be interested in Hebigo. It's an alternative reader for Hissp implementing a language designed to resemble Python. It's not exactly Python, but it's so close that a regular Python user could probably pick it up pretty quickly. Its macro suite has the alternatives that a reverse transpiler would need for most of the Python language. Paredit and friends don't work on Hebigo, but Hebigo is regular enough that a Paredit-style structural editing package would theoretically be possible, and is designed in such a way that you probably don't need one: you can rearrange things almost as well yourself mostly by copy-pasting blocks of lines and fixing indents, features already available in fairly dumb editors. Reverse transpiling Hissp to Hebigo is much more doable than to Python, although it might not always come out formatted the way you'd like (which is probably the job of a separate formatter). Neither of these tools have been implemented.

And finally, check out py2hy. Impressive work so far, but neither complete nor up to date, last I checked.

[jleonard-r7]

I think the way to go is going to be to (structurally) edit the s-expression formulation of Python AST. At least one of those projects has the code for AST -> s-expression. So I would just need to isolate that and then find a way to go from AST back to Python source code (which I imagine already exists somewhere in the Python standard library). Any pointers on these two possibilities would be appreciated, thank you.

It occurs also to me that this is a pattern that should be repeatable for all possible languages & I’m surprised really that emacs or the Lisp community doesn’t have the infrastructure already in place for that. If it were in place, Lisp could truly never die (and no one would have to know that we are using it).

[gilch]

The standard library has ast.unparse. I think there are also third-party alternatives with extra features.

[gilch]

You might also be interested in projectional editing, like in MPS.

[jleonard-r7]

Thank you!