Skip to content

Commit 976d31f

Browse files
committedFeb 24, 2013
Fixed tokenization of "a%1"
Updated failed tokenization examples to use bad octal digits Conflicts: History.rdoc test/test_rdoc_markup_to_html.rb test/test_rdoc_markup_to_html_snippet.rb
1 parent 8eb2ae2 commit 976d31f

File tree

4 files changed

+22
-4
lines changed

4 files changed

+22
-4
lines changed
 

‎History.rdoc

+1
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
Ruby Bug #6488 by Benny Lyne Amorsen.
66
* Fixed lexing of character syntax (<code>?x</code>). Reported by Xavier
77
Noria.
8+
* Fixed tokenization of % when it is not followed by a $-string type
89

910
=== 3.12.1 / 2013-02-05
1011

‎lib/rdoc/ruby_lex.rb

+1-1
Original file line numberDiff line numberDiff line change
@@ -1034,7 +1034,7 @@ def identify_quotation
10341034
type = nil
10351035
lt = "\""
10361036
else
1037-
raise Error, "unknown type of %string #{type.inspect}"
1037+
return Token(TkMOD, '%')
10381038
end
10391039
# if ch !~ /\W/
10401040
# ungetc

‎test/test_rdoc_markup_to_html_snippet.rb

+5-3
Original file line numberDiff line numberDiff line change
@@ -379,19 +379,21 @@ def test_accept_verbatim_ruby_error
379379
rdoc.options = options
380380
RDoc::RDoc.current = rdoc
381381

382-
verb = @RM::Verbatim.new("a %z'foo' # => blah\n")
382+
verb = @RM::Verbatim.new("a % 09 # => blah\n")
383383

384384
@to.start_accepting
385385
@to.accept_verbatim verb
386386

387+
inner = CGI.escapeHTML "a % 09 # => blah"
388+
387389
expected = <<-EXPECTED
388390
389-
<pre>a %z'foo' # =&gt; blah
391+
<pre>a % 09 # =&gt; blah
390392
</pre>
391393
EXPECTED
392394

393395
assert_equal expected, @to.res.join
394-
assert_equal 19, @to.characters
396+
assert_equal 16, @to.characters
395397
end
396398

397399
def test_add_paragraph

‎test/test_rdoc_ruby_lex.rb

+15
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,21 @@ def test_class_tokenize_heredoc_percent_N
105105
assert_equal expected, tokens
106106
end
107107

108+
def test_class_tokenize_percent_1
109+
tokens = RDoc::RubyLex.tokenize 'v%10==10', nil
110+
111+
expected = [
112+
@TK::TkIDENTIFIER.new(0, 1, 0, 'v'),
113+
@TK::TkMOD.new( 1, 1, 1, '%'),
114+
@TK::TkINTEGER.new( 2, 1, 2, '10'),
115+
@TK::TkEQ.new( 4, 1, 4, '=='),
116+
@TK::TkINTEGER.new( 6, 1, 6, '10'),
117+
@TK::TkNL.new( 8, 1, 8, "\n"),
118+
]
119+
120+
assert_equal expected, tokens
121+
end
122+
108123
def test_class_tokenize_percent_r
109124
tokens = RDoc::RubyLex.tokenize '%r[hi]', nil
110125

0 commit comments

Comments
 (0)