Why using map combinator on delegate excludes it from list of tokens?

[gumanoid]

This code successfully parses any text:

object Wut : Grammar<List<String>>() {
    val literal by regexToken(".{1,3}")
    override val rootParser by oneOrMore(literal use { text })
}

However, if I move use { text } to definition of literal:

object Wut : Grammar<List<String>>() {
    val literal by regexToken(".{1,3}") use { text }
    override val rootParser by oneOrMore(literal)
}

it complains "the token list should not be empty".

Why position of use { text } matters? For me this seems to break referential transparrency.

I'm also puzzled why token list is "empty" in second case? I've explicitly told it to use literal

[h0tk3y]

Defining tokens as separate declarations inside the grammar is needed for collecting the tokens and building a tokenizer on top of them. If you use tokens "inline" in other parsers, the grammar's current implementation won't be able to register the tokens and use them in a tokenizer.

So, the workaround is to always declare tokens as delegated properties inside a grammar.

Given that the set of tokens and their order really matters in parsing, following the convention of defining tokens as delegated properties also makes it more clear as to what tokens are there and in which order.

The grammar implementation could potentially be improved in some ways that would make best effort to enforce registering tokens as properties.

If you want, you can customize the logic for registering tokens in your grammar and override the grammar's val tokens: List<Token> so that it returns the actual set of the tokens that it needs to use in tokenizing the input.

[gumanoid]

First of all, thanks for clarifications.
I've overlooked that *token functions are returning Token, and map combinator returns Parser

Token order really matters to parsing, indeed. But I doubt that it should be mapped to order of field declarations. Order of field/method declarations inside of a class/object, in general, can be arbitrary and should not affect code behavior. Thus it is unexpected to average programmer that field reordering could break something.
The concept of ordering maps naturally to List<T>. For me, the code that explicitly declares list of tokens in order, is much clearer. Maybe overriding val tokens: List<Token> should be mandatory just like rootParser?

[gumanoid]

One more thing that I can't get is: parser combinators is purely functional concept. Why mess with OOP world here by leveraging delegation at all?