String parsing incorrect

[triska]

Currently, I get:

$ scryer-prolog
?- use_module(library(lists)).
   true.
?- length("\x0\\x0\", L).
   L = 1, unexpected.
   L = 2. % expected

[triska]

Also:

?- "\x0\\x0\" = [_,_].
   false, unexpected.
   true. % expected

[UWN]

?- "\x0\" = [Null].
   Null = '\x0\'. % ok
?- "ab\x0\" = [A,B,Null].
   false, unexpected.
   A = a, B = b, Null = '\x0\'.
?- "ab\x0\" = [A,B].
   A = a, B = b, unexpected.
   false. % expected, but not found

[triska]

This issue may be confined to parsing strings, at least the following works exactly as expected:

?- A = '\x0\\x0\', atom_chars(A, Cs), Cs = [_,_].
   A = '\x0\\x0\', Cs = "\x0\\x0\".

So, 0-bytes are correctly handled in this case.

[UWN]

So in a (single) quoted token (* 6.4.2 *) the null character is handled correctly whereas in a double quoted list token (* 6.4.6 *) it is ignored, but only if double quotes denote a list of characters! So it is not the parsing process as such which is incorrect, but rather this conversion.

?- set_prolog_flag(double_quotes,chars).
   true.
?- Cs = "a\x0\b".
   Cs = "ab", unexpected.
?- set_prolog_flag(double_quotes,codes).
   true.
?- Cs = "a\x0\b".
   Cs = [97,0,98].
?- set_prolog_flag(double_quotes,atom).
   true.
?- Cs = "a\x0\b".
   Cs = 'a\x0\b'.

[bakaq]

[...] So it is not the parsing process as such which is incorrect, but rather this conversion.

Nah, I've dealt with the parser before (#2254) and I'm pretty sure this is just a parsing problem. The parser deals with the double_quotes at parsing time. This seems like an easy fix actually, and I think I just found the place of the problem. Will report the findings if I find anything.

[bakaq]

Yeah, I solved it. Sending a PR in a moment.

[triska]

Thank you a lot, this is solved in rebis-dev and can be closed once rebis-dev is merged into master!