[tables] Avoid whitespace around float placement by disallowing float pages

[jensmaurer]

This is particularly noticeable for Table 1 (basic character set).

[tkoeppe]

Have a look at the grammar immediately following the heading of 5.4 [lex.pptoken], that's now orphaned.

Table 1 used to just be on a page of floats, didn't it? It's not that there's "unnecessary whitespace"; it's just that the float goes on its own page because there wasn't space to fit it on an existing page. I don't think that's particularly bad?

[jensmaurer]

I understand that the heading [lex.pptoken] is now separated from the rest of its contents, but that's independent of the issue. Since grammar definitions can't be split in the middle, LaTeX decides to split after the heading.

If there were a way to force-prohibit splitting after a heading, I'd be all for it, but it seems there are two (!) entries in the TeX vertical list that allow splitting here (penalty=0), so there's not much of a chance to get that right. I wouldn't be surprised if the bnflist would add such an entry, because splitting in front of a chunk of grammar seems rather ok-ish.

Pertinent data from \tracingpages=1; comments from me:

% t=592.3503 plus 24.0 minus 4.0 g=706.0 b=10000 p=-300 c=100000#
---> Just before the heading. We like splitting here (penalty=-300), but 592pt is so far from the goal of 706pt, the badness is 10000.
\openout5 = `std.xtr'.

 (./std.xtr
---> writes the index entry at the start of [lex.pptoken]
% t=616.3823 plus 28.3055 minus 4.86108 g=706.0 b=3168 p=0 c=3168#
% t=616.3823 plus 28.3055 minus 4.86108 g=706.0 b=3168 p=0 c=3168#
---> these two are after the heading, before the grammar chunk. Cost=3168 is quite bad, but it's the best we had so far.
% t=760.9503 plus 28.3055 minus 4.86108 g=706.0 b=* p=-51 c=*
---> after the grammar chunk, which doesn't fit on the page anymore.

ISO/CS complained that the table was centered on the page of floats, as opposed to starting at the top. And there's still lots of whitespace on the page that could be filled by the text that follows; it's not that there would be more tables to come. So, why have a page of floats to start with?
(It turns out we lose an entire page for the full document due to the bad situation around Table 1.)

[jensmaurer]

Ok, here is what happens with the stray page break in front of grammar snippets:

We have our special "light-gray" coloring for grammar non-terminals. This causes the following on the vertical list (output from \showboxdepth=4 and \showboxbreadth=1000):

....\write6{\indexentry[generalindex]{token!preprocessing|(hyperpage}{\thepage
}}
....\penalty 10000
....\penalty 10000
....\penalty 10000
....\pdfcolorstack 0 push {0.2 g 0.2 G}
....\glue 0.0
....\pdfcolorstack 0 push {0.2 g 0.2 G}
....\glue(\parskip) 4.3055
....\glue(\parskip) 0.0
....\glue(\baselineskip) 2.01251
....\hbox(6.25+1.75)x418.99994, glue set 365.00966fil, shifted 50.00006 []

Note the \pdfcolorstack entries, which separate the \glue entries from the \penalty entries. Which means the (sequences of) \glue entries are now (each) a page breaking opportunity. If I remove \color{grammar-gray} from the definition of \BnfNontermshape, those \pdfcolorstack entries go away, and the page break after the heading is avoided reliably (checked by looking at the PDF):

....\write6{\indexentry[generalindex]{token!preprocessing|(hyperpage}{\thepage 
}}
....\penalty 10000
....\penalty 10000
....\penalty 10000
....\glue 0.0
....\glue(\parskip) 4.3055
....\glue(\parskip) 0.0
....\glue(\baselineskip) 2.01251
....\hbox(6.25+1.75)x418.99994, glue set 365.00966fil, shifted 50.00006 []

[jensmaurer]

Manually adding a penalty after the color setting "fixes" the issue:

....\penalty 10000
....\pdfcolorstack 0 push {0.2 g 0.2 G}
....\penalty 10000
....\glue 0.0
....\pdfcolorstack 0 push {0.2 g 0.2 G}
....\penalty 10000
....\glue(\parskip) 4.3055

[jensmaurer]

@tkoeppe , I've attached a patch for the fix.

[tkoeppe]

That's great, thank you!

[tkoeppe]

The fix is great, and I've added that to the main branch. I'm not yet sure about the page-of-floats change; I'd be OK leaving that to the IS for the time being.

[AlisdairM]

As this seems relevant for the main branch, but not the pending IS, I would note that my attempt to reflow the whole of [lex] to follow the phases of translation happens to solve many of the problems with floating tables floating onto following page and into the wrong subsections: #7193

I have rebased and verified line-by-line that the copy/paste text of the reordering is identical to the current main branch, and this might be worth a review once the current review priorities have passed.