VALUE and TO

[andrewtholt]

Hi,

For my education and amusement I'm trying to figure out how VALUE and TO are implemented.

e.g.

0 value fred
cr fred . 

1 to fred
cr fred . 

: tst 2 to fred ;

cr tst fred . 

So when I run these on gforth this is what I see:

0 value fred  ok
cr fred .  
0  ok
  ok
1 to fred  ok
cr fred .  
1  ok
  ok
: tst 2 to fred ;  ok
  ok
cr tst fred .  
2  ok

So my question is about the behavior of 'to' I'm guessing that it uses state to determine if compiling or interpreting.

If interpreting then it replaces the value assigned to fred
If compiling it places the execution tokens that replace the value.

I've played around with this and can't get it work as expected.

Any advice, tips, examples appreciated.

Thanks.

[alexshpilkin]

Maybe I’m misunderstanding your question (this is really the kind of thing that’s best to discuss synchronously, say on IRC...), but what you’re describing is indeed the correct approach (inside the bounds of ANS), so I’m not sure what it is that I can help you with. As you say, TO must execute in either compilation or interpretation state (because it has to parse the name from the input stream either way), thus must be immediate (in a standard program). However, other than that, it has to perform two different things depending on STATE.

For reference, this kind of thing is known as a “STATE-smart word”, and it has been the subject of much flames and motivated much of recent compiler work (dual-xt words, recognizers, etc.; for an early discussion see Ertl’s EuroForth ’98 paper). One problem is that STATE-smart words are essentially impossible to reliably POSTPONE, and indeed it’s not entirely clear what it means to do so. ANS Forth usually tries to avoid STATE-smartness (e.g. .( is separate from ."), but somehow it prescribes it for TO (and forbids using POSTPONE and friends on it).

Aside from these STATE issues, I expect reimplementing VALUE and TO is not a problem for you, so here’s an interpretation-only version for reference:

: my-value   create , does> @ ;
: my-to   ' >body ! ;

My recommendation is to now try and make a compilation-only version. What should it do? Obviously it has to be immediate. The tick must stay as is, because it must parse during compilation; the store must be postponed, because it must be performed during execution. >BODY can be done whenever. What probably tripped you up is that tick will leave a value on the stack during compilation, while store needs it to be on the stack during execution; thus something must be done to compile that value into the word currently being compiled. (I find it easiest to debug this kind of thing using a non-destructive stack print such as .S.) Afterwards, merging the two versions using STATE @ IF is completely straightforward.

I think I’ve said as much as I can without spoiling the whole thing for you, so I’ll hide the complete implementation under a spoiler.

Solution

: my-value   create , does> @ ;
: my-to   ' >body  state @ if postpone literal postpone ! else ! then ; immediate

0 my-value fred   cr fred . ( 0 )
1 my-to fred   cr fred . ( 1 )
: tst 2 my-to fred ;   cr tst fred . ( 2 )

[ruv]

One problem is that STATE-smart words are essentially impossible to reliably POSTPONE,

Sure, it's possible. Please take a look at my post How POSTPONE should work.

[ruv]

Just several comments-corrections to show a slightly wider picture.

So my question is about the behavior of to. I'm guessing that it uses state to determine if compiling or interpreting.

It's not necessary.

A number of different implementations for to is possible, and they are connected to the corresponding implementations for value.

Roughly, these implementations can be separated to the following approaches:

1. to is immediate and STATE-dependent;
2. to is STATE-independent (at all), but it changes some internal state;
3. find (or its alternative) is STATE-dependent for to (this way is always specific for a particular system).

If interpreting then it replaces the value assigned to fred
If compiling it places the execution tokens that replace the value.

This phrasing can be correct in the approach (1) only. In the approach (2) "it" always does the same. In the approach (3) "it" references the different subroutines in the first and in the second place, and neither of them depends on STATE.

[mitra42]

Here's how I implement it in webForth  ( https://github.com/mitra42/webforth ) I use one code word `>BODY` that knows how to find the body or a word from its xt, since my code is token threaded it uses token stored at the XT to determine if it was defined with `CONSTANT`, `VARIABLE`, `USER`, `DEFER` etc this is code because depending on the type of word it might be looking in Ram or Rom or initialized User variable space. Then I can build the whole ANS mess of `DEFER`, `TO`, `IS`, `VALUE`, `ACTION-OF` (my least favorite part of ANS on top of that word. Note that I support ANS's `POSTPONE` I still use `[COMPILE]` and `COMPILE` as its much easier for me to understand the code when I do ! ``` : >BODY! \\ ( Store a value in the body of the variable ) >BODY ! ; : DEFER! >BODY! ; ( to be honest I cant remember why this a seperate word from >BODY! - maybe historical reason ) : '>BODY! STATE @ ( nasty state dependent word ) IF [COMPILE] ['] COMPILE >BODY! ELSE ' >BODY! THEN ; : IS '>BODY! ; IMMEDIATE ( _USER is a variable that points to the next USER variable available, $,n is from eForth, builds next dictionary word from a string ) : VALUE TOKEN $,n OVERT HERE tokenValue , _USER @ , _USER ++ >BODY! ; : TO '>BODY! ; IMMEDIATE : DEFER@ ( xt -- a; ) >BODY @ ; : ACTION-OF STATE @ \ https://forth-standard.org/standard/core/ACTION-OF (nasty state dependent word) IF [COMPILE] ['] COMPILE DEFER@ ELSE ' DEFER@ THEN ; IMMEDIATE ``` \- Mitra

…

On 15/10/21 3:30 am, ruv wrote: Just several comments-corrections to show a slightly wider picture.

[pahihu]

( A.Haley interpreted VALUEs --- 16sep2021 ) : NOOP ; VARIABLE #MSG 0 MSG ! CREATE VALUES ‘ @ , ( <value>) ‘ ! , ( TO <value>) ' +! , ( +TO <value>) ' NOOP , ( 'OF <value>) : VALUE ( n "name" -- ) CREATE , DOES> VALUES #MSG @ CELLS + @ EXECUTE 0 #MSG ! ; : TO ( "name" -- ) 1 #MSG ! ; : +TO ( "name" -- ) 2 #MSG ! ; : 'OF ( "name" -- ) 3 #MSG ! ; \ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pahihu

…

On 2021. Oct 14., at 12:56, Andrew Holt ***@***.***> wrote: Hi, For my education and amusement I'm trying to figure out how VALUE and TO are implemented. e.g. 0 value fred cr fred . 1 to fred cr fred . : tst 2 to fred ; cr tst fred . So when I run these on gforth this is what I see: 0 value fred ok cr fred . 0 ok ok 1 to fred ok cr fred . 1 ok ok : tst 2 to fred ; ok ok cr tst fred . 2 ok So my question is about the behavior of 'to' I'm guessing that it uses state to determine if compiling or interpreting. If interpreting then it replaces the value assigned to fred If compiling it places the execution tokens that replace the value. I've played around with this and can't get it work as expected. Any advice, tips, examples appreciated. Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#108>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTEB23H4KGTOTGONGBAZ5TUG2ZPFANCNFSM5F7LPVQQ>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[ruv]

: to 1 msg# ! >in @ parse-name 2drop >in ! ; immediate

You know, this definition operationally equivalent to the original one. The fake parse does nothing observable.

[gerryjackson]

VFX has implemented TO and friends as immediate messages for 20+ years.
: to 1 msg# ! ; immediate
The handler (VALUE for example) then selects the appropriate execution or compilation action. Because VFX handles non-default compilation semantics correctly (see recent EuroForth papers) there's no non-standard behaviour. The only weasel activity is the that the the implementation of TO and friends depends on the 'AS IF' rule because it behaves as if the system parsed.

A definition such as this works for (silly but disproves the 'AS IF' claim) code such as (tested in VFX Forth):

0 value x 0 value y  ok
1 to to to to x  ok
x  .  1  ok

to  2  x  ok
x  .  2  ok

3  0  to  [if]  x  [else]  y  [then]   ok
x . 2  ok
y . 3  ok

-1 to 4 swap [if] x [else] drop 5 y [then]   ok
x  .  4  ok
y  .  3  ok

: foo {: x :} x x . ;
98 99 to foo 98  ok

In fact any amount of Forth code between the TO and its target, possibly in included files etc.

All these are ambiguous conditions (therefore not portable) according to the specification of TO so it doesn't matter, but I'd rather ambiguous conditions are detected by a system if possible.

[spelc]

As I get older, I have more and more sympathy with Chuck's view that one should not code for situations that may potentially exist. Coding for silly billy mistakes is reasonable, coding for nasal daemons is not.

[AntonErtl]

Concerning nasal demons, they are actually an expression used by C compiler writers to defend a position similar to the one you are exhibiting here: If a progam exhibits undefined behaviour, the C compiler is allowed to have "demons fly out of your nose."

I have claimed that Forth compiler writers are saner. Was I wrong?

[spelc]

It is certainly true that the AS IF rule is just a weasel way of making a long-standing implementation technique "sort-of" standard. The trouble is that we (MPE) support a dozen or more operators for various defining words such as VALUE and to assist the ClassVFX notation. The message number idea goes back all the way to the original Bartholdi implementations of VALUE.

The question for me is whether this particular standard non-compliance actually matters. Non-compliance reported by users as a technical support issue matters a lot. What is being reported here is that separating to operator ( TO ) from the child of VALUE is non-standard. I agree. I think that the discussion is more about the better error reporting that a parsing implementation can/should implement.

However, the whole point of the message number technique is that it separates (disconnects) the operator from the VALUE child. It is quite feasible to implement a midway technique that parses to
check if the target word is one that uses the TO mechanism. How much better this is than the existing check in ; that the operator has been consumed (easily fooled) is doubtful, and I am loath to add this sort of code of little utility.

As to sanity, I have made a living from Forth for 40+ years.

[bfox9900]

This how I added VALUE and TO to Camel Forth.
Some people object to the use of STATE smart words but I am not "state smart enough" to avoid them. :)

\ VALUEs for CAMEL99
\ Usage:  0 VALUE W
\           99 TO W   1 +TO W
HERE
 .( .)
: VALUE   CONSTANT ;
: TO  ( n -- )
           '  >BODY   \ compute PFA at compile time
           STATE @
           IF   POSTPONE LITERAL  POSTPONE !  EXIT
           THEN  ! ; IMMEDIATE
 .( .)
\ comment this out if you don't need it
: +TO      ( n -- )
           ' >BODY
           STATE @
           IF   POSTPONE LITERAL  POSTPONE +!  EXIT
           THEN  +! ; IMMEDIATE
 .( .)
HERE SWAP -  SPACE DECIMAL . .( bytes) HEX

[ruv]

This how I added VALUE and TO to Camel Forth.

Your implementation is almost portable . A one exception is VALUE that relies on a specific implementation of CONSTANT, without any necessity. A better implementation could be:

: VALUE  ( x "name" -- ) CREATE , DOES> @ ;

Rationale: a system can be optimized in such a way that constants are compiled as literals, and after that your definition for TO will not work anymore.

Also, it should be noted that this TO is standard compliant only if the Locals word set is not provided by the system. Since this TO (and +TO) cannot be applied to local variables, when a standard TO should be applicable to locals.

Some people object to the use of STATE smart words but I am not "state smart enough" to avoid them. :)

Just redefine POSTPONE to make it correctly work for immediate STATE-dependent words. In any case, it works incorrectly in CamelForth, e.g. POSTPONE LITERAL produces a non compliant code, as I can see (1, 2).

A correct implementation for POSTPONE is as follows:

[UNDEFINED] LIT, [IF]
  : LIT, ( x -- ) COMPILE LIT  ,  ;
[THEN]
: EXECUTE-COMPILING ( i*x xt -- j*x )
  STATE @ IF  EXECUTE  EXIT  THEN
  -1 STATE !  EXECUTE  0 STATE !
;
: POSTPONE ( "name" -- )
  BL WORD FIND DUP 0= ?ERR 1 =  ( xt flag-special )
  IF ['] EXECUTE-COMPILING ELSE ['] COMPILE, THEN ( xt xt-compiler )
  STATE @ IF SWAP LIT, COMPILE, EXIT THEN EXECUTE
; IMMEDIATE

(It's an adaptation of my portable implementation to CamelForth CAMEL99-V2)

[bfox9900]

Yes you are of course correct.
The TI-99 system has 24K of contiguous memory :-) so I save the space.

[pahihu]

For more examples see T.Rayburn: “METHODS> Object-oriented extensions redux”, FORML 1987 (Vierte Dimension Volume III/Nr.3 Oktober 1987) P.Bartholdi: “The ‘TO’ solution”, Forth Dimensions Volume 1, Number 4 pahihu

…

On 2021. Nov 11., at 14:59, theBF ***@***.***> wrote: This is very clever. More of an OOP approach with the message interpreter. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#108 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTEB24HCIIDQAMRJPXDJTLULPD23ANCNFSM5F7LPVQQ>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.