Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse without allowing ignored tokens #26

Open
rnett opened this issue Jun 8, 2020 · 2 comments
Open

Parse without allowing ignored tokens #26

rnett opened this issue Jun 8, 2020 · 2 comments

Comments

@rnett
Copy link

rnett commented Jun 8, 2020

I want to allow whitespace most of the time, so I have token("\\s+", true). However, in a few scenarios, I want to disallow whitespace between tokens/subexpressions. A relatively easy way to do this would be nice.

The best for my use would be a version of */and that doesn't ignore anything.

@h0tk3y
Copy link
Owner

h0tk3y commented Jun 8, 2020

Normally this is implemented as so-called 'lexer modes' which the lexer switches upon encountering a token of a specific type. A popular example is switching to the comment tokenizing mode for the rest of the line when a // occurs.

I do have plans to implement tokenizing modes in better-parse when I have more time.

@rnett Would anything like that cover your use case?

@rnett
Copy link
Author

rnett commented Jun 8, 2020

Possibly? It would be overkill by quite a bit though. I'm trying to parse dice, e.g. 5d12, and want 5d12 to parse but not 5 d 12. I'm just using another token for it at the moment and it works. Another example would be not allowing spaced before the left paren in function calls, e.g. foo(1) but not foo (1).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants