Elaborate on all phases of CTL implementation to encourage collaboration

[luzpaz]

• Phase 1 & Phase 2 will be worked on simultaneously (rationale: Elaborate on all phases of CTL implementation to encourage collaboration #3 (comment)). Phase 1: Make PDF production (+ PS production + svg-export) glyph based instead of character based. Phase 2: Separate StoryText from Layout (milestone)
  • class StoryText stores only the unicode, charstyle and special items like paragraph separators, inline objects and footnotes
  • a new class TextLayout stores the glyphs, their relations to the unicode text and their visual positioning
  • the visual positioning is represented by a tree of boxes, which come in several flavors: class GlyphBox for a sequence of glyphs which all have the same font and formatting, class LineBox for a sequence of glyphboxes, class BlockBox for a sequence of lineboxes, class InlineBox for embedded pageitems and other boxes for more fancy layout (if you know TeX you'll get the picture)
  • class TextLayout will also provide all methods for cursor positioning (next line, end of column, selection, screen coordinate to StoryText position mapping, ...)
  • Boxes will have a method render(ScPainterEx* painter)
  • The interface ScPainterEx will have a method drawGlyphs(const GlyphBox& glyphrun)
  • PS and PDF export will be implementations of the ScPainterEx interface.
• Phase 3: Integrate Harfbuzz (milestone) is an OpenType layout library. By using Harfbuzz we will enable features like:
  • Ligatures and language specific layouts.
  • With some luck we will even enable Indic shaping.
• Phase 4: Implement a box-oriented layout mechanism (milestone), i.e. store the layout as a hierarchy of boxes. Like in TeX and HTML, boxes can be arbitrarily nested.
  • Refactor TextLayout to store a hierarchy of boxes: GlyphBoxes store text, HBoxes store lines of text, VBoxes store paragraphs or similar structures.
  • Rendering code has to be adapted in order to handle this flexibility.
• Phase 5: Integrate Fribidi (milestone) is a library for layouting RTL (right-to-left) languages like Arabic, Hebrew and Persian.
  • Elaboration needed
• Phase 6: Expand the style system with CSS-like features
  • Elaboration needed

[ftake]

I think an important issue is that Scribus does not support embedding (subsets of) Japanese fonts. I guess it also does not Chinese ones.

[luzpaz]

@ftake, yes, the plan is to have CJK support as well. I don't know which library is related to that. I'm still waiting on @andreas-vox to help me organize this issue so I will offer guidance to developers

[luzpaz]

@ftake a forum post pointed me to this: https://sites.google.com/a/borndigital.jp/scribus/scribus-en
looks like a patch to support Top to Bottom text for Scribus 1.4

[ftake]

Thanks, there seems to be many things to do. I have to know the inside of cairo to port this feature into Scribus.

By the way, the problem (and the patch above) is not related to top to bottom layout (aka. vertical writing). That is also an interesting (but heavy) challenge.

[andreas-vox]

Ok, here's some elaboration on steps 1 and 2:

The old Scribus way to store text is a long list of characters for each story (represented as C++ class StoryText):

1: unicode char | charstyle | glyphlayout | more glyphlayouts if necessary ...
2: unicode char | charstyle | glyphlayout | more glyphlayouts if necessary ...
3: unicode char | charstyle | glyphlayout | more glyphlayouts if necessary ...
4: unicode char | charstyle | glyphlayout | more glyphlayouts if necessary ...
5: unicode char | charstyle | glyphlayout | more glyphlayouts if necessary ...
...
Each line is represented by the soon-to-be-obsolete class ScText (this is the result type of nearly-obsolete method StoryText::item(int pos) ).
Class GlyphLayout contains a glyph ID and relative positioning/scaling for this glyph.

This design has several drawbacks:

• there's a 1:n relation from unicode characters to glyphs, i.e. each glyph needs a unicode char
• the order of glyphs is tied to the logical order of characters
• charstyle information is repeated for each character (not critical to CTL but bad design)
• no hierarchical structure of blocks and paragraphs

So the new way I'm aiming at looks like this:

• class StoryText stores only the unicode, charstyle and special items like paragraph separators, inline objects and footnotes
• a new class TextLayout stores the glyphs, their relations to the unicode text and their visual positioning
• the visual positioning is represented by a tree of boxes, which come in several flavors: class GlyphBox for a sequence of glyphs which all have the same font and formatting, class LineBox for a sequence of glyphboxes, class BlockBox for a sequence of lineboxes, class InlineBox for embedded pageitems and other boxes for more fancy layout (if you know TeX you'll get the picture)
• class TextLayout will also provide all methods for cursor positioning (next line, end of column, selection, screen coordinate to StoryText position mapping, ...)
• Boxes will have a method render(ScPainterEx* painter)
• The interface ScPainterEx will have a method drawGlyphs(const GlyphBox& glyphrun)
• PS and PDF export will be implementations of the ScPainterEx interface.

The last three points require some heavy lifting with the ScPainterEx interface, so for now I'll concentrate on the text part with the class GlyphRun which will be what's passed to PS and PDF export instead of the old ScText structure.

[luzpaz]

@andreas-vox thanks for the elaborations. I've updated the top comment of this thread.

[luzpaz]

Discussion ongoing in "ScribusCTL" threads:
http://lists.scribus.net/pipermail/scribus-dev/2015-November/thread.html#start
http://lists.scribus.net/pipermail/scribus-dev/2015-December/thread.html#start

[luzpaz]

@dawoodalbadi would you be so kind to help me understand where we're at in regards to milestones reached with the ScribusCTL so far? Thanks!

[dawoodalbadi]

Hello Now we implementing boxes method and we still in this stage but we almost done. Now scribus store glyphs layout in tree of boxes and the code has been tested for pdf production and it work fine with only one small bug(repeating one or two letters at the end of line). We modified the code only for text frame and I need later to modify it to deal with path text also. I have to say if boxes method done completely correct it will be easy for us to integrate harfbuzz and freebidi in scribus and that basically CTL support Regards

[luzpaz]

@dawoodalbadi do you mind checking off (or even modifying by adding/removing subtasks) from Step 1 & 2 (AKA Phase 1 & 2) from #3 (comment)