Interface proposal

[jcubic]

First thing that we need to agree before adding Scheme interpreter code and front-end REPL. In need to be generic and allow to change Scheme implementation and maybe also the REPL.

In my interpreter I have two ports stdout nad stdin in global scope that was used to write and read and also interpreter exposed eval and function that could be called from JavaScript. That was my main interface. In recent version I've hidden those ports because user was able to overwrite those names.

stdout, stdin and stderr need be hidden, but I'm not sure how this would look like in Gambit.

The other idea is to make eval and read JavaScript functions that call scheme functions.

There is one issue that I'm worried about (but it's working in gambit-in-browser) that read need to be async function. It can't block the thread like Emscripten default behavior (I'm not sure if it changed, but c read function invoked prompt ugly window). In my interpreter I've solved the issue by making everything is async. read return JavaScript promise and eval may or may not return a promise (exec main API always do that), but promises are hidden from the users. They are resolved automagically.

I was looking into gambit-in-the-browser intf.js file and I've seen some setTimeouts loops, I'm not sure if would be e

Just my ideas, you have any please write them below.

[jcubic]

from @feeley in mailing list.

I’ve added some files to the try.scheme.org github repo to show how a barebones Scheme interpreter webapp can be created using Gambit. It is a makefile and a 45 line Scheme source file.

The interpreter is only “barebones” in its UI. It currently runs a REPL using the JavaScript “prompt” function that allows showing the last few lines of output and allowing the next line of input to be entered. Very minimal but often good enough to start developing webapps.

The I/O for the interpreter’s console can be redirected elsewhere by redefining the JavaScript functions g_console_input_add and g_console_output_add. I’m confident an interface to JQueryTerminal would be easy to achieve with minimal work. For Gambit-in-the-browser I had used Terminus (https://github.com/breuleux/terminus) another JavaScript terminal emulator,with lots of interesting features by the way. I expect JQueryTerminal to be similarly easy to interface to.

File I/O would have to be improved to support editing files in the browser. Currently open-input-file will use AJAX to read files from the web server, which is useful for accessing libraries and other resources.

[jcubic]

Great starting point. I've stared writing issue on GitHub but I didn't add it,
I'm not sure if should. I was proposing adding proper API that should be
simple that can be connecting Scheme implementation with REPL.

To add I/O files we can use BrowserFS I was planing on adding this to my
Scheme implementation, I have issue created. I can try to think something
about this.

The one issue with I/O in Emscripten is prompt for read from stdin. I was
impress that your gambit in browser support real (read) in terminal. It's not
something easy to create and replace the default prompt that is blocking and
replace with async code.

I can check the code you've added, but probably will need help to make the
(read) async to add it to jQuery Terminal. With LIPS I was using Promises
that is best way to make sync code.

Gambit in browser have:

_user_interrupt
TTY.register(FS.makedev(6, 0), ops);

_heartbeat_interrupt();
_cleanup();

and setTimeout

That don't look very promising in creating simple API, but maybe something
can be added and make simple API. that will be more generic and allow to
change the implementation.

[feeley]

Having non-blocking input (for the console but also ajax, websockets, etc) is really important to me and is why this is implemented in the Gambit-in-the-browser webapp. It is a required feature for allowing multiple Scheme threads to do I/O without blocking other threads. This has been working in Gambit for many years.

When Scheme code is compiled to JavaScript by Gambit the runtime system provides a trampoline that allows leaving the Scheme code to give time for the browser to process events, then the trampoline can be called again to resume the execution of the Scheme code. The API needs to provide this functionality.

Gambit's JavaScript runtime system has a class that represents a "device" (the G_Device class). Moreover each device has a "condition variable" for blocking threads that try to get data from a device and the data is not yet available. So the I/O subsystem can keep track of which condition variable needs to be "signaled" when an I/O operation is now possible on that device and this will cause the waiting thread to be resumed by the Scheme thread scheduler. The thread will attempt the same operation again, and this will succeed (or not)... allowing the data to be processed by the thread.

Here's for example the function for reading data from a device:

g_os_device_stream_read = function (dev_condvar_scm, buffer_scm, lo_scm, hi_scm) { ... };

All the parameters are Scheme objects. The first is the device and condition variable, the second is the u8vector buffer for getting the data, and finally there are lo and hi indices of the section of the buffer that receives the data. The function returns the number of bytes read with the special case 0 for end-of-file, and -35 for "operation would block" (EAGAIN if you know Unix system calls).

The details are rather Gambit specific, but the general idea could be used to create a general purpose API that can be applied to other Scheme implementations including Gambit.

The important point is that the underlying I/O subsystem (in JavaScript) can keep track of the set of objects (dev_condvar_scm in the above example) that need to receive the notifications that the I/O operation should be attempted once more. The details of the notification would be specific to the Scheme implementation. A Scheme implementation that doesn't want to bother with the notifications (and threads, etc) could simply retry the I/O operation after a short delay to basically implement a busy-wait.

[lassik]

Any new thoughts on this issue?

[jcubic]

This is explanation part of the Gambit system but I have no clue how this is related to the code in https://feeley.github.io/gambit-in-the-browser/intf.js I don't see any g_os_device_stream_read in that file so I don't see how this is related.
Is _user_interrupt function defined by user coder code and related to device.
Do we need to defined devices for stdin stdout stderr? I don't understand how this works so I'm not able to implement the interface for Gambit to be used with jQuery Terminal.

I would need to see example that is non blocking. prompt based code is like any other emscripten compilation and it's not even close to solution done for gambit-in-browser.

From what I see it's really expensive in CPU to run Gambit in browser, because it run in the setTimeout loop even if it do nothing. It consume 19% of CPU even that it do nothing. It do complex computation when idle. I'm not sure how it would work on mobile, and yes applications run on mobile and I think try.scheme.org should also work on mobile. People may want to run mobile version to test something. My REPL works fine on Android.

To think about, it's in 2020 and people still care if they make size of the page smaller by 50KB. I've tested the gambit in browser in lighthouse (integrated with Chrome dev tools) and performance is 5% which is less then really bad.

[jcubic]

Maybe it would be good idea to first write documentation how GambitJS works and how to use it. Maybe it would be good idea to write article "how to use Gambit Scheme in browser". Since there not much pages in https://www.google.com/search?hl=pl&q=scheme%20in%20browser.

The only result is BiwaScheme and no one know any other option of Scheme in browser. So if Gambit can run it browser maybe it's worth explaining how to use it.

[feeley]

The gambit-in-the-browser system is based on the normal "C" distribution of Gambit compiled to JS by emscripten. This was done a few years ago (Gambit v4.7.0) and things have evolved quite a bit since then (Gambit is now at v4.9.3).

The more interesting option for try.scheme.org is the version of Gambit compiled to JS using the "universal" backend. It has a tighter coupling to JS and doesn't depend on emscripten, so it is more compact and has better performance. I'm not sure what you are talking about when you say that Gambit's performance is 5%. In any case if you care about performance you should read my "Compiling for multi-language task migration" paper which shows that Gambit-JS is up to 2 to 3 orders of magnitude (100x to 1000x) faster than Spock and Scheme2JS, the two other fairly mature Scheme to JS compilers.

Gambit JS can definitely run on mobile platforms. What I was saying in a previous message is that mobile platforms should not be the main target users for try.scheme.org . The user experience will be suboptimal because of the UI constraints (hard to use virtual keyboard and small screen size on phones).

I fear that using a minimalistic Scheme interpreter for try.scheme.org will not give a good impression of Scheme's abilities and power. Here are the features I think are essential in the try.scheme.org REPL to allow users to get a good understanding of what Scheme is:

• Close to R7RS conformance (100% conformance is too much to ask because very few systems are at 100% currently)
• A debugger with good error messages, a backtrace and single-stepping
• Tail-calls
• First-class continuations
• define-syntax + syntax-rules
• define-library
• Bignums, rationals, complex numbers

Gambit JS has all of these requirements and many other features, including threads, hash-tables, pretty printer, JS FFI, fast execution, and the ability to compile Scheme apps to JS and run them in the browser or nodejs. These advanced features are also interesting to show off in examples even if they are not essential features.

I think we should agree on the requirements of the try.scheme.org REPL before moving forward. Do you agree with the above requirements?

[lassik]

Gambit-JS is 2 to 3 orders of magnitude (100x to 1000x) faster than Spock and Scheme2JS, the two other fairly mature Scheme to JS compilers.

Awesome job!

Here are the features I think are essential in the try.scheme.org REPL

Definitely can't hurt to have them.

• 100% conformance is too much to ask because very few systems are at 100% currently

As far as I know, there are none. Chibi-Scheme may be closest, but it's interpreter-only, needs to be compiled with Emscripten and doesn't have a debugger.

[arthurgleckler]

On Sat, Jan 2, 2021 at 3:20 PM Lassi Kortela ***@***.***> wrote: Here are the features I think are essential in the try.scheme.org REPL Definitely can't hurt to have them.

I agree with Marc. If we're trying to show off the language, we should insist upon as complete an implementation of one of the recent standards, i.e. R6RS or R7RS, as possible.

[lassik]

I think we should give every implementation space on try.scheme.org that can reasonably run in the browser. But the default choice should be something full-featured and performant.

As for which order to do things in, I'm a believer in letting the people who do the work pick the order.

[feeley]

@lassik it may be "politically correct" to make space for multiple Scheme's on try.scheme.org . However if try.scheme.org is more than a link to a Scheme REPL running in the browser, such as having specific examples and a tutorial, then it becomes a mess to limit examples to the lowest common denominator or to explain in the tutorial that the following example only runs on Scheme X.

My point of view is that try.scheme.org will become the principal way to give a first impression of Scheme to newcomers. Giving them the impression that there are many many incompatible versions of Scheme is not the best image to leave them with, even if it is reality.

[lassik]

Unfortunately, one of the main goals with scheme.org has to be figuring out how to avoid political splits. I'm optimistic that a good compromise between usability and political stability can be found.

I agree that it's best if try.scheme.org immediately brings up the default REPL, and the other REPLs are in a pull-down menu or something.

It's worth thinking about how to organize tutorials and other documentation with respect to the REPLs. It's useful to have a REPL at hand while reading documentation. But maybe the REPL shouldn't be visible or loaded by default in that case; the user would have to click a button to bring it up or something -- people consulting reference documentation likely already have a REPL running in Emacs or a terminal window.

Anyway, we should definitely have try.scheme.org as something that immediately offers a REPL, but it probably is useful if we can embed REPLs into other parts of scheme.org as well.

[arthurgleckler]

This may not be a good idea, but I'll bring it up just in case, since it was popular in the early days of the web: There's no particular reason that try.scheme.org has to run Scheme directly in the browser. If it's suitably sandboxed and rate-limited, we could run a Scheme REPL on a server and provide remote access to it through the browser. That would significantly reduce client-side overhead, and would allow us to run full-fledged Schemes at full speed. I'll also note that the startup repl.it supports Scheme, and that they do it using BiwaScheme, so there's already a minimal Scheme available for people to try for free. That means that there's less pressure on us to deploy something that's just a subset of Scheme.

…

[lassik]

A server-side REPL is definitely possible, but is a substantial performance and security concern.

@feeley pointed out that Emscripten should be able to compile many Schemes for use in the browser with reasonably few modifications. With modern browsers, that could be a more promising approach to REPLs for many of them.

For the default REPL, a JS-native Scheme would probably be better than an Emscripten-based one.

[feeley]

@arthurgleckler I would advise against a server-side REPL... a big can of worms. A web based Scheme REPL can be feature rich, be usable offline, have a persistent file system, and have reasonably good performance while avoiding security issues and logins to some form of "user accounts".

I'm not against supporting multiple Scheme systems in the long term. In the short term however it adds another level of complexity to make it easy to add a JavaScript Scheme implementation as a "plugin" (because existing JS Scheme implementations may not have a design suitable for this).

One interface that works well for Gambit JS is a hook in the stdin and stdout ports. These ports are used for the REPL I/O (reading expressions, writing results and error messages and reading debugger commands). The user's typing events feed the stdin port, and the output to stdout feeds the "console" (which could be a plain "textarea", a CodeMirror instance, a ymacs instance, etc). So the API is essentially at the stream level.

This may not be suitable for JavaScript Scheme implementations designed around an "eval" function which receives a string and returns a string (the result), and raises a JS exception when the evaluation causes an error. With this sort of API it is much harder to have a feature rich debugger, because it needs its own debugger API (how to display error messages, how to show backtraces, how to continue or single step the code, etc).

So I propose starting with a stream based API (basically hooks into stdin/stdout) and those JS Scheme implementations with stream based REPLs should be adaptable relatively easily to be used as plugins to try.scheme.org .

[feeley]

@feeley pointed out that Emscripten should be able to compile many Schemes for use in the browser with reasonably few modifications. With modern browsers, that could be a more promising approach to REPLs for many of them.

I didn't quite say that... I said that "in principle" any Scheme implementation written in portable C code should be compilable to JS by emscripten. However, in practice it may be a very different story (in part because it is common practice to use C tricks to implement some things and these might ruin portability to emscripten). You really have to try it for each Scheme implementation and either it works out of the box (very unlikely) or there is some adaptation required (that may take hours to months of work to implement).

For example, emscripten used to compile Gambit fine in the past (including January 2020), but I just tried again a few days ago and the C compiler crashes when compiling Gambit (due to some bug in LLVM). I managed to avoid the bug by compiling with -O0 optimization level, but haven't resolved other issues and don't have much time to spend on this (I was just interested in doing a quick check).

[jcubic]

BTW: 5% Performance is from lighthouse tool in Chrome that is used to test few metrics about the website:

The tools say that most people will abandon the website because of performance (but that's just estimation).
This performance is for mobile, which is lighthouse default. Desktop test is little higher at 15%.

Note that this metrics is not about Gambit itself only about the website gambit in browser.

[lassik]

@jcubic Interesting, thanks for computing those metrics! feeley.github.io runs the Emscripten version of Gambit. I wonder what Lighthouse would say about the gsc -target js version.

[lassik]

I didn't quite say that... I said that "in principle" any Scheme implementation written in portable C code should be compilable to JS by emscripten. However, in practice it may be a very different story (in part because it is common practice to use C tricks to implement some things and these might ruin portability to emscripten).

Sorry about misquoting you.

It seems this discussion is converging on the issue of defining a common JavaScript interface to which different Scheme implementations can conform to show a REPL. If we can define such an interface, that works well for Gambit and Lips to start with, then we could leave it as other implementors' responsibility to support that interface if they want to add their REPL.

[lassik]

Ideally the common interface would be something that can be supported by an Emscripten-based implementation as well. I haven't looked at what kind of JS FFI Emscripten has.

[feeley]

A stream based interface seems a natural fit for the traditional Scheme interpreter REPL "read-eval-print" metaphor, where the read is from stdin and the print is to stdout. The rest of the human interface (for debugging) is text based. So this would allow all non-JS Scheme implementations to be adapted to work (by compiling with emscripten), and also Gambit-JS, but I'm not sure about the other JS Scheme implementations.

[feeley]

In addition to a stdin/stdout interface it would be good to support a user-interrupt (ctrl-C) to interrupt the execution. This will be useful especially for newcomers that create infinite loops without knowing it.

[jcubic]

@feeley is it possible to make CTRL+C break loop in Gambit? It would be awesome if it do. I wonder if I could add something like this to my interpreter.
As for common interface it's mostly because it would be easy to integrate with jQuery Terminal. Right now don't care much try.scheme.org will work with LIPS I only would like to implement all the features that I already have in my REPL. like colors, auto indent, pretty printing on copy paste or completion. Probably not all features would be able to create in GambitJS like in LIPS can get all functions from environment and can use it create completion and each procedure/macro have doc string in code and I can show it to the users when he hover over procedure or macro. If gambit don't store that information maybe we can use docs from somewhere else. maybe docs.scheme.org.

[feeley]

Yes this is implemented in the gambit-in-the-browser REPL. Here's an example interrupting a long recursive computation and asking for a backtrace using the REPL's ",b" command:

This can also be made to work with the Gambit JS system (it is a question of implementing the UI).

As for documentation, Gambit has the help procedure to open the documentation in the browser. This works in the normal Gambit system if you want to try it out. It used to work in the gambit-in-the-browser REPL, but unfortunately the URL of the Gambit documentation is no longer valid so you will get a 404 error.

In what language do you propose implementing the features you describe (auto-indent, pretty-printing, auto-completion)? I suspect you have JS in mind but it would be nice to implement this in Scheme (for example as a module loaded by the Scheme implementation) so that it is well integrated to the specific Scheme implementation being used. For example Gambit already has a pretty-printer with specific layout style and also symbol completion and it would be good for the UI to use the existing conventions of the Scheme implementation.

[feeley]

Let me also mention here my codeBoot project which is an online IDE for JavaScript and Python which supports single stepping. It is possible to create signed URLs which start codeBoot in a given state (content of file editors and state of execution). This provides a nice way to refer to specific code examples in web pages, PDF documents, etc. Here's an example of a small Python program being single-stepped (click the "play 1" button to advance by one step): example.py

I'll be adding support for Scheme (based on Gambit JS). So perhaps this would be another way to offer an online Scheme REPL.

Your thoughts?

[jcubic]

@feeley looks great. The stepper is pretty awesome. Is it full python or some proof of concept (because in REPL if I call again load smae file it throw error that load is not defined, and yield throws syntax error). There is one existing Python in JS which is Brython that is very good, I have simple REPL with syntax highlighting at trypython.jcubic.pl but it don't have lot of features like my Scheme REPL (the latest one is at https://jcubic.github.io/lips-test/ it's WIP for new website, the REPL on the page have little bit of lag, because the code is written in Scheme and it wait for Scheme to load the std library, maybe I will try to bundle JS and Scheme files together, so it will load faster).

[feeley]

codeBoot is a WIP and is designed for teaching programming to novices, so it covers a subset of Python. Nevertheless the subset is quite rich including classes and exceptions and there's also an interface to the DOM.

The load(...) at the REPL is a fake operation that is used to indicate that a program was executed at that point (using the "play" button while the focus is in a code editor window). You can use the Python import ... statement if you want to load a file.

Concerning the REPL interface I have an idea how to structure the system. We could use two JS classes: Console and Interpreter. The Console has a method to "connect" it to an Interpreter so that user interaction at the console will send notifications to the connected interpreter using these public methods:

• interp.console_readable(cons) is called when the console has some data to read
• interp.console_interrupted(cons) is called when ctrl-C is typed at the console

A Console maintains a queue of lines that were entered by the user. A Console has these public methods:

• cons.read() returns the next line in the queue including the \n or an empty string if an "end-of-file" is generated at the console with ctrl-D or null if there is nothing in the queue
• cons.write(text) displays the text at the console

So an Interpreter's console_readable(cons) method is called any time some data is added to the connected Console's queue. The Interpreter can then retrieve the data from the console by calling the read() method, either immediately or sometime later. It is important to decouple the notification of available data and the actual reading of the data because the Interpreter may be busy at that moment.

The interface could eliminate the Console's read() method and do the queuing in the Interpreter. However I like that the Console has both a read and write method because that is closer to the way TTY's are implemented in Unix (so the interface might be easier to use for other Scheme implementations that have a TTY interface).

Here's how a Console and Interpreter can be created and connected:

function init() {
    var cons = new Console(document.querySelector('#repl'));
    var interp = new Interpreter();
    cons.connect(interp);
}

The advantage of a separate connect method is that it allows multiple Consoles to feed a single Interpreter, which might be useful for a page with multiple examples. It could also be used to dynamically change which Console is connected to which Interpreter.

[jcubic]

Two questions:

• What about if user use (read) and he type "xxx yyy" Scheme read is expression based and console is line based will there be no problem? It would be great if this works like in normal REPL, where it read xxx and yyy is sent to Scheme.

Example in Kawa:

#|kawa:1|# (read)
#|kawa:2|# xxx (+ 1 2)
xxx3

Gambit do similar:

> (read)
xxx (+ 1 2)
xxx
> 3

Or will Scheme (GambitJS) just get the line and do what is should?

• Will there be some kind of pause of console (I have this in my jQuery Terminal) so when Scheme is executing the code, will Console keep accepting the line as input? Or is this also handled by the Scheme (GambitJS)?

[feeley]

I think it would be good to implement the same "stream" interface as when a Scheme interpreter is executed from the unix shell (i.e. what you show above). That way newcomers will get the same experience when they move to a "desktop" environment and also it makes it more likely that a non-JS implementation of Scheme can be adapted to the try.scheme.org REPL.

I have a prototype implementation of the Console class based on CodeMirror (~ 250 lines of code). How hard would it be to write a version of the Console class based on JQuery Terminal?

[jcubic]

You can upload the files to some directory in git I can check how it works with CodeMirror. If you have something that works without prompt it probably will be easy to ad jQuery Terminal integration.

I probably will need help if we will want to implement all the features I have in my REPL. They can be written in Scheme:
The features that wound be nice to have:

• Completion with function/macro names (it will require more code to make it work with scope like (let ((foo-bar 10)) (+ fo| TAB, it don't even need to evaluate the expression it can just inspect the tree and find few expressions like let, this code can be written in Scheme). Does Gambit have function that return all defined names from environment?
• Pretty Print on copy/paste - in my REPL I have pretty printing (if you copy from R7RS pdf all indent is removed) also I have breaking rules when user copy/paste single line of code.
• Syntax highlighting is just single line of JS.
• Help message will require to write simplified version of docs that will have signature and few lines of description. Maybe we can parse docs for Gambit and fetch first paragraph of docs. I don't know how docs looks like because the site is dead (at list when I've last checked).
• Auto indent and parenthesis matching (I have tokenizer function that return all parenthesis I can reuse that JS code I have, unless you would like to have Scheme code for that).

[arthurgleckler]

On Thu, Mar 25, 2021 at 6:24 PM Marc Feeley ***@***.***> wrote: Here's a small section of VM.min.js (part of the JS code generated for fib) to give you a taste :

I'm sure you spotted the obvious bug in that code. Alas, I don't have room in this message to elaborate.

[feeley]

The code is much easier to read at lower compactness levels! The first 2 lines of the code above are equivalent to the following code generated at -compactness 0 :

_bb1_fib = function () {// entry-point
  if (_nargs !== 1) {
    return _wrong_nargs(_bb1_fib);
  }
  if (typeof _r1 === "number") {
    if (_r1 < 2) {
      return _bb33_fib();
    } else {
      return _bb5_fib();
    }
  } else {
    return _bb14_fib();
  }
};

_bb1_fib.id = 0;
_bb1_fib.parent = _bb1_fib;
_bb1_fib.nfree = -1;
_bb1_fib._name = _make_interned_symbol("fib");
_bb1_fib.ctrlpts = null;
_bb1_fib.info = false;

_bb4_fib = function () {
  if (_r1 < 2) {
    return _r0;
  } else {
    return _bb5_fib();
  }
};

_bb4_fib.id = 1;
_bb4_fib.parent = _bb1_fib;

_bb33_fib = function () {
  return _r0;
};

_bb33_fib.id = 2;
_bb33_fib.parent = _bb1_fib;

[lassik]

I have a new version up on https://gambitscheme.org/try . I have added a -compactness LEVEL option to gsc to control the use of various strategies to reduce the amount of generated code. With -compactness 9 the file VM.min.js.gz is a mere 890KB (down from 1.8MB) and the file VM.min.js is 7MB (down from 24MB).

Fantastic, great news!

[gambiteer]

I get the message The certificate for gambitscheme.org expired on 4/3/2021.

[jcubic]

@feeley Demo mode is awesome. I thought that the example were missing at first. Maybe there should be a way to trigger the demo and that it's documented. It's kind of hidden and work like magic (not documented at all, which is bad).

[feeley]

It's been a long time since the last update... I have now added a file editor to the user inferface and updated to the latest Gambit version. The demo shows off a few more tricks. On try.scheme.org I get a CORS related error while executing code that fetches a document from github. When I execute this from a local web server all is fine. If you know how to fix it that would be great!

[jcubic]

It's not CORS error, it's CSP (Content-Security-Policy) that is used by the website. I think it's for all subdomains on scheme.org. I've already complained about this in mailing list, because I can't use my Scheme REPL bookmark on any of the scheme.org websites.

[arthurgleckler]

On Tue, Dec 28, 2021 at 2:20 AM Jakub T. Jankiewicz < ***@***.***> wrote: It's not CORS error, it's CSP (Content-Security-Policy) that is used by the website. I think it's for all subdomains on scheme.org. I've already complained about this in mailing list, because I can't use my Scheme REPL bookmark <https://lips.js.org/#bookmark> on any of the scheme.org websites. Here's the earlier discussion:

https://srfi-email.schemers.org/schemeorg/msg/18298964/ I'm not sure what to do about this. Does anyone understand the security implications of leaving this header out on this page?

…

Message ID: ***@***.***>

[jcubic]

I think for this particular case (of scheme.org) it doesn't make any sense to even add CSP header. You don't provide any input from users. Those are static html pages. Only really big website add CSP like YouTube or GitHub where XSS may be a problem. I would say that there are 0% possibility of any security implication. And if you need to control other subdomains that will not be static html pages you can add CSP there.

I would understand having it on Planet Scheme, where there is user content.

Maybe the solution is white list or black list of domains that need to have CPS. If page is dynamic and render user content CSP is reasonable, but not on static HTML pages.

CPS is mostly for XSS protection and there're no risk if there're no user input and pages are not webapps.

[arthurgleckler]

On Tue, Dec 28, 2021 at 9:14 AM Jakub T. Jankiewicz < ***@***.***> wrote: I think for this particular case (of scheme.org) it doesn't make any sense to even add CSP header. You don't provide any input from users. Those are static html pages. Only really big website add CSP like YouTube or GitHub where XSS may be a problem. I would say that there are 0% possibility of any security implication. And if you need to control other subdomains that will not be static html pages you can add CSP there. I would understand having it on Planet Scheme, where there is user content.

Yes, but we're talking about try.scheme.org, which definitely does have input from users — in the form of code. I don't understand the full security implications of putting something like that up on the web with unrestricted ability to have it fetch from other sources.

…

Message ID: ***@***.***>

[feeley]

I think what would be problematic is if the server was storing user input. In the case of try.scheme.org it is the client that stores user input (the files of the filesystem implemented with browser local storage, and the REPL history). I think that enabling CSP essentially means "content obtained from try.scheme.org can be trusted". This would not be OK if users were allowed to add arbitrary documents (i.e. malicious code) to the server. In the case of the web REPL, users can only store things in their own browsers.

[arthurgleckler]

On Tue, Dec 28, 2021 at 7:05 PM Marc Feeley ***@***.***> wrote: I think what would be problematic is if the *server* was storing user input. In the case of try.scheme.org it is the *client* that stores user input (the files of the filesystem implemented with browser local storage, and the REPL history). I think that enabling CSP essentially means "content obtained from try.scheme.org can be trusted". This would not be OK if users were allowed to add arbitrary documents (i.e. malicious code) to the server. In the case of the web REPL, users can only store things in their own browsers. That sounds reasonable.

I'll try to figure out the configuration-generating code in https://github.com/schemeorg/linux-configurations soon and see what I can do. However, given the holidays, it may take me a while. Adding Lassi since he wrote that code (which looks well factored, by the way).

…

Message ID: ***@***.***>

[jcubic]

try.scheme.org get input from users but it's all in JavaScript, XSS is a problem when it's saved on the server or reflected on the page (like executing user code from URL). WHen user (attacker) will be able to craft the url that will execute Scheme code, he will be able to trick the user to run XSS, but note that on try.scheme.org there are no sensitive information to be stolen. XSS is a problem when there are user data that can be stolen.

Also If code is executed from URL CSP will not protect against it. But note that there are no information to be stolen even if you can execute Scheme code from URL. It would only be problem if Code can get outside the browser window/tab sandbox.

[feeley]

I confirm that the try.scheme.org web REPL does not execute code embedded in the URL because this would cause security issues: it would allow a malicious person to create a URL that, when visited by an unsuspecting user, would read the local storage of the user's web REPL (the files he has created in the browser and the REPL history) and send them to anywhere on the internet. The expected privacy of the data entered by the user when using try.scheme.org would be violated.

So I don't see any security issues with enabling CSP on try.scheme.org, and there are benefits in enabling CSP such as reading data and also executable code from third party web servers. The later would be useful for a user to import Scheme modules that are not centrally managed and vetted by a globally trusted authority (i.e. that manages scheme.org). It is only required that the user trust the third party. A compelling case is the user trusting that his github account contains trusted code so that he can commit new code to github and then import it in the web REPL. The web REPL has its own whitelist mechanism to control access to external sources. A user confirmation is requested when accessing data and code from a source not on the whitelist, for example:

Adding a trusted source to the whitelist (to avoid user confirmations) can be done like this: \_os_whitelist_add("https://raw.githubusercontent.com/feeley/"). This interface is a bit awkward but it could be wrapped in a more Schemey API when more experience is gained on the use cases.

In conclusion: please enable CSP on try.scheme.org !

@jcubic Thanks for helping me better understand the difference between CORS and CSP. Here's my mental model... please correct me if I'm wrong... There are 3 parties involved in the use case: the web REPL app running in the user's web browser, the try.scheme.org web server that sent the web REPL app to the browser, and some other web server (X) that contains data the user wants to access using the web REPL app.

• CSP is for the try.scheme.org server to authorize the web REPL app to access data on X.
• CORS is for X to authorize the web REPL app to access data on X (knowing that the app came from try.scheme.org).

So both are required for accessing external data.

[jcubic]

@feeley

In conclusion: please enable CSP on try.scheme.org !

I think you mean please disable CSP, it's not like CORS when you need to enable it, it's enabled right now, that's why you can't load 3rd party code. I think that CSP can be removed (on try.scheme.org) without causing any security issues, as you said.

[feeley]

Yes of course!

[feeley]

To rephrase my mental model:

There are 3 parties involved in the use case: the web REPL app running in the user's web browser, the try.scheme.org web server that sent the web REPL app to the browser, and some other web server (X) that contains data the user wants to access using the web REPL app.

• CSP is for the try.scheme.org server to forbid the web REPL app to access data on X.
• CORS is for X to authorize the web REPL app to access data on X (knowing that the app came from try.scheme.org).

[jcubic]

If think that there should be repo that will contain whitelist of sites that don't need CSP and that it can be disabled for them (most security people will say that whitelist is always better than a blacklist, it's better to have CSP enabled by default and disable it for some of the pages). But I personally would add to the list every subdomain that are static html page, CSP was created for server side web applications. But there may be a reason for some pages to have CSP even if they are static html pages, I personally don't know any reason.

But unfortunately don't know how to set it up on Linux server.

[arthurgleckler]

On Wed, Dec 29, 2021 at 8:42 AM Jakub T. Jankiewicz < ***@***.***> wrote: If think that there should be repo that will contain whitelist of sites that don't need CSP and that it can be disabled for them. I personally would add to the list every subdomain that are static html page, CSP was created for server side web applications. But there may be a reason for some pages to have CSP even if they are static html pages, I personally don't know any reason. Lassi (CC-ed) created such a repo:

https://github.com/schemeorg/linux-configurations. It will have to be altered to make it drop CSP entirely for a domain, but it's close, and the code is clean. I'm not sure how to put it into production yet, though. I hope to have some time to look at it next week, but if someone else wants to send a pull request, I'd be grateful.

…

Message ID: ***@***.***>

[feeley]

New version of the web REPL consistent with Gambit v4.9.4 has been put on try.scheme.org .

[arthurgleckler]

On Thu, Dec 30, 2021 at 8:22 AM Arthur A. Gleckler ***@***.***> wrote: Lassi (CC-ed) created such a repo: https://github.com/schemeorg/linux-configurations. It will have to be altered to make it drop CSP entirely for a domain, but it's close, and the code is clean. I'm not sure how to put it into production yet, though. I hope to have some time to look at it next week, but if someone else wants to send a pull request, I'd be grateful.

I've finally had time to look at this, but I haven't yet been able to get the Ansible-based code to work to reproduce the existing state. Once I figure that out, I can try to update the generated Nginx configuration to drop those headers. Any help would be welcome.

…

Message ID: ***@***.***> >

[arthurgleckler]

On Wed, Jan 5, 2022 at 10:19 PM Arthur A. Gleckler ***@***.***> wrote: I've finally had time to look at this, but I haven't yet been able to get the Ansible-based code to work to reproduce the existing state. Once I figure that out, I can try to update the generated Nginx configuration to drop those headers.

I've updated the Nginx configuration for try.scheme.org to omit the Content-Security-Policy header. It turns out that Lassi already automated the whole process through schemeorg.sh <https://github.com/schemeorg/linux-configurations/blob/master/schemeorg.sh>, which I had used before. I was confused about the relationship between this script and other code used for scheme.org production. I had to fix some conflicts between the server's current configuration and the generated Ansible scripts, but updating the Scheme code to remove the header was easy. *Would you please check to see whether this helps?* I'm sorry for the delay. By the way, if anyone has heard from Lassi, please let me know. I haven't been able to reach him since November.

…

Message ID: ***@***.***> >> >

[lassik]

We just confirmed with Arthur that Content-Security-Policy is now omitted on try.scheme.org.

[feeley]

Great! I have tried try.scheme.org and I notice that the (import (srfi 28)) that is executed as part of the demo takes several seconds to execute and this causes the demo's simulated input to get out of sync. From what I can see the modules are being interpreted rather than using the compiled version. I will investigate later.

[feeley]

I have started working on a tutorial for try.scheme.org (and try.gambitscheme.org). My WIP allows the file editor area on the right side of the UI to open .html files and display them as HTML by default and in text format by using a menu. Below you can see the look of the file TUTORIAL/basic.html which is the first page of the tutorial. The buttons at the top help navigate to the different sections of the tutorial. Each code example has a "run" button to execute that code by injecting appropriate Scheme evaluations to the REPL at the left.

Do you have any suggestions on the look before I continue working on the other sections? Would anyone care to help out with writing the other sections that are just HTML? The "run" buttons simply have anonclick handler that sends commands to the REPL. For example:

<p>The traditional Hello World example can be done from the REPL by calling the <code>display</code> procedure as shown here:</p>
<pre class="g-code-example"><button onclick="UI.find(event.target).demo([0, '(display &quot;hello world!\\n&quot;)\n']);">run</button>&gt; <b>(display "hello world!\n")</b>
hello world!
</pre>

[jcubic]

Look nice, it matches the whole application. You can check my Try JavaScript website for inspiration. This is how I would code this if I would need to create something like this myself. I used an interface of a fake browser that changes the URL when you navigate the links on the page. To make this work I needed to use a Proxy because normally you can access the URL inside an iframe if the page has a different origin (prototcol+domain+port).

[feeley]

I have uploaded a new UI to try.scheme.org . The new UI puts the tutorial in a section to the right of the editor area so that it is possible to click on links in the tutorial to edit files in the editor area. There are 5 sections in the tutorial that cover the basics of Scheme and a section on writing and using libraries. I have a few examples of importing libraries directly from github repositories without a prior installation.

Please read through the tutorial and let me know if there are sections that could be improved.

[arthurgleckler]

On Mon, Oct 24, 2022 at 10:21 PM Marc Feeley ***@***.***> wrote: Please read through the tutorial and let me know if there are sections that could be improved.

Very nice. I have a few copy edits and one trivial editorial comment: Basic - Fix agreement: "A wide range of characters *are* allowed" ⇒ "A wide range of characters *is* allowed" Forms - Fix spelling: "mentionned" ⇒ "mentioned" - Add comma: "bindings and the expressions are evaluated from left to right" ⇒ "bindings, and the expressions are evaluated from left to right" - Add comma: "in which case they are implicitly wrapped in a begin form." ⇒ ", in which case they are implicitly wrapped in a begin form." - case: Explain that case uses eqv? for comparison. Libraries - Use correct article and add comma: "To use the (hello) library a (import (hello)) form" ⇒ "To use the (hello) library, *an* (import (hello)) form" - It's strange that the extension to the import syntax doesn't include the URI scheme: (import (github.com/gambit/hello))

[feeley]

Thanks @arthurgleckler for the comments! Should be fixed soon.

Question to native speakers: is it better to say "allowing easy publication of new libraries" or "allowing easy publishing of new libraries"?

Concerning your last comment I prefer the terse syntax because you always want to use https (it would be a security disaster to fetch code using http!). Web browsers are intelligent enough to figure out what a URI refers to without the URI scheme. GoLang also drops the URI scheme from module names, surely for "elegance" of the notation.

Feel free to suggest new sections for the tutorial. There's certainly things to be said on tail calls, continuations, files and other aspects of Scheme and I ran out of steam after several days of hacking at this (not just the text but all the plumbing for clicking on examples to execute them in the REPL). Here's an example showing the construction of the HTML code using JavaScript code:

'<p>At the REPL the traditional Hello World can be done as shown below by calling the <code><b>display</b></code> <i>procedure</i> (a term preferred to <i>function</i> by Schemers).</p>\n\
' + runnable_repl_example('\
> (display "hello world!\\n")\n\
hello world!\n\
') + '\n\
<p>Procedure calls follow the parenthesized syntax\n\
<pre>    (<i>&lt;procedure&gt;</i> <i>&lt;argument1&gt;</i> <i>&lt;argument2&gt;</i> ...)</pre>\n\
and even simple arithmetic operations like addition and multiplication are procedure calls:</p>\n\
' + runnable_repl_example('\
> (+ 6 1)\n\
7\n\
> (expt 2 8)\n\
256\n\
> (* 4 (atan 1))\n\
3.141592653589793\n\
')

[gambiteer]

I would say "making it easy to use existing libraries and publish new ones".