Compute TokenList.value dynamically (v2) #710
Open
+52
−28
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR supersedes #623. The meat of the PR is the same: fix the remaining portion of issue #621 by making
TokenList.value
a dynamically-computed property rather than an attribute. This avoids the quadratic runtime behavior that occurred due to recomputingTokenList.value
each timeTokenList.group_tokens(
) was called withextend=True
.The previous PR #623 had some rather awkward hacks related to stripping comments, but I found that I could avoid those by simply tweaking the comment stripping process to strip comments from a token list before stripping any sublists, making this PR much simpler.