Skip to content

scanner: emit BAD-STRING and keep scanning - #26

Open
team-humaki wants to merge 1 commit into
gorilla:mainfrom
team-humaki:scanner-bad-string
Open

team-humaki wants to merge 1 commit into
gorilla:mainfrom
team-humaki:scanner-bad-string

Conversation

@team-humaki

Copy link
Copy Markdown

Fixes #12.

Unclosed " / ' currently latch Scanner.err as TokenError ("unclosed quotation mark"), so the rest of the stylesheet is never tokenized. CSS Syntax 3 consume-a-string instead yields a <bad-string-token>:

  • unescaped newline → BAD-STRING, reconsume the newline, continue
  • EOF → BAD-STRING

Add TokenBadString and emit it for the consumed text without setting Scanner.err.

Unclosed quotes are a <bad-string-token>, not a fatal Scanner.err.
An unescaped newline ends the bad string and is reconsumed so later
rules still tokenize.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parser does not properly emit bad-string tokens

1 participant