Incrementally render by sending the operator list by chunks as they're ready. #3461

brendandahl · 2013-07-10T23:30:58Z

To make this much easier I stripped out much of uneeded promise code for getting operator lists.

I've created RenderTask/InternalRenderTask to help with rendering that make it much easier to cancel the rendering and resume when more chunks are ready. I don't really like extending the Promise for page.render(), but this way we don't break anyone's code. I've also created an OperatorList class so we have one central one that we can flush as it is built. This also cleans up a lot of code since we had little pseudo operator list objects in a number of places.

Things we should further explore:

Chunk size - played a bit not sure what's optimal yet.
Render skipping images if they are ready, then restarting when they are.
On the main thread empty the operator list as it comes in, instead of appending it. This would help with memory but complicates things if we have multiple renderTasks going at once.

Should also note for complex pdfs this drops the rendering time quite a bit on my machine. For instance:
http://www.sfbike.org/download/map.pdf
Old code: nothing appears for 15s, 28s to finish rendering
New: appears ~2s, 15s to finish rendering

brendandahl · 2013-07-10T23:31:29Z

/botio test

pdfjsbot · 2013-07-10T23:31:31Z

From: Bot.io (Windows)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://107.22.172.223:8877/b5ffa5c04ca1bd9/output.txt

pdfjsbot · 2013-07-10T23:31:31Z

From: Bot.io (Linux)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://107.21.233.14:8877/568cd594709cb8d/output.txt

pdfjsbot · 2013-07-10T23:55:51Z

From: Bot.io (Linux)

Failed

Full output at http://107.21.233.14:8877/568cd594709cb8d/output.txt

Total script time: 24.34 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.21.233.14:8877/568cd594709cb8d/reftest-analyzer.xhtml#web=eq.log

pdfjsbot · 2013-07-11T00:00:29Z

From: Bot.io (Windows)

Failed

Full output at http://107.22.172.223:8877/b5ffa5c04ca1bd9/output.txt

Total script time: 28.95 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.22.172.223:8877/b5ffa5c04ca1bd9/reftest-analyzer.xhtml#web=eq.log

Snuffleupagus · 2013-07-11T12:43:28Z

@brendandahl This is a really impressive performance improvement!

If I understand it correctly, this PR supersedes #3164.

brendandahl · 2013-07-12T22:31:06Z

/botio test

pdfjsbot · 2013-07-12T22:31:08Z

From: Bot.io (Linux)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://107.21.233.14:8877/69d02d7c5d405d8/output.txt

pdfjsbot · 2013-07-12T22:31:08Z

From: Bot.io (Windows)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://107.22.172.223:8877/af25a5bcae60b65/output.txt

pdfjsbot · 2013-07-12T22:54:53Z

From: Bot.io (Linux)

Failed

Full output at http://107.21.233.14:8877/69d02d7c5d405d8/output.txt

Total script time: 23.75 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.21.233.14:8877/69d02d7c5d405d8/reftest-analyzer.xhtml#web=eq.log

pdfjsbot · 2013-07-12T22:59:14Z

From: Bot.io (Windows)

Failed

Full output at http://107.22.172.223:8877/af25a5bcae60b65/output.txt

Total script time: 28.19 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.22.172.223:8877/af25a5bcae60b65/reftest-analyzer.xhtml#web=eq.log

brendandahl · 2013-07-16T22:28:12Z

Waiting for #3477, then I'll update this.

timvandermeij · 2013-07-18T14:17:51Z

@brendandahl Sounds like a great optimization! Can't wait to test out a preview. #3477 has been merged.

brendandahl · 2013-07-18T20:12:59Z

/botio test

pdfjsbot · 2013-07-18T20:13:01Z

From: Bot.io (Windows)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://107.22.172.223:8877/2b0994cb1fcfad2/output.txt

pdfjsbot · 2013-07-18T20:13:02Z

From: Bot.io (Linux)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://107.21.233.14:8877/c275bea24571421/output.txt

pdfjsbot · 2013-07-18T20:37:30Z

From: Bot.io (Linux)

Failed

Full output at http://107.21.233.14:8877/c275bea24571421/output.txt

Total script time: 24.48 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.21.233.14:8877/c275bea24571421/reftest-analyzer.xhtml#web=eq.log

pdfjsbot · 2013-07-18T20:41:21Z

From: Bot.io (Windows)

Failed

Full output at http://107.22.172.223:8877/2b0994cb1fcfad2/output.txt

Total script time: 28.24 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.22.172.223:8877/2b0994cb1fcfad2/reftest-analyzer.xhtml#web=eq.log

yurydelendik · 2013-07-31T13:12:11Z

src/api.js

-        function pageStartRenderingFromOperatorListEnsureFonts() {
-          self.displayReadyPromise.resolve();
+      function complete(error) {
+        for (var i = 0; i < self.renderTasks.length; i++) {


I think var i = self.renderTasks.indexOf(internalRenderTask); if (i >= 0) ... will look better

yurydelendik · 2013-07-31T13:58:42Z

Looks good. We have to start playing with it more. Landing time?

/botio-windows preview

pdfjsbot · 2013-07-31T13:58:45Z

From: Bot.io (Windows)

Received

Command cmd_preview from @yurydelendik received. Current queue size: 0

Live output at: http://107.22.172.223:8877/b6ca2f8455ffea7/output.txt

pdfjsbot · 2013-07-31T13:59:03Z

From: Bot.io (Windows)

Success

Full output at http://107.22.172.223:8877/b6ca2f8455ffea7/output.txt

Total script time: 0.29 mins

Published

Viewer: http://107.22.172.223:8877/b6ca2f8455ffea7/web/viewer.html
Extension: http://107.22.172.223:8877/b6ca2f8455ffea7/extensions/firefox/pdf.js.xpi
Extension (AMO): http://107.22.172.223:8877/b6ca2f8455ffea7/extensions/firefox/pdf.js.amo.xpi

…e ready.

yurydelendik · 2013-07-31T20:08:33Z

/botio test

pdfjsbot · 2013-07-31T20:08:35Z

From: Bot.io (Linux)

Received

Command cmd_test from @yurydelendik received. Current queue size: 0

Live output at: http://107.21.233.14:8877/77a8fbae21c47e9/output.txt

pdfjsbot · 2013-07-31T20:08:35Z

From: Bot.io (Windows)

Received

Command cmd_test from @yurydelendik received. Current queue size: 0

Live output at: http://107.22.172.223:8877/2eb7fda7021a849/output.txt

pdfjsbot · 2013-07-31T20:31:37Z

From: Bot.io (Linux)

Failed

Full output at http://107.21.233.14:8877/77a8fbae21c47e9/output.txt

Total script time: 23.03 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.21.233.14:8877/77a8fbae21c47e9/reftest-analyzer.xhtml#web=eq.log

pdfjsbot · 2013-07-31T20:37:15Z

From: Bot.io (Windows)

Failed

Full output at http://107.22.172.223:8877/2eb7fda7021a849/output.txt

Total script time: 28.61 mins

Font tests: Passed
Unit tests: Passed
Regression tests: FAILED

Image differences available at: http://107.22.172.223:8877/2eb7fda7021a849/reftest-analyzer.xhtml#web=eq.log

yurydelendik · 2013-07-31T20:46:11Z

/botio makeref

pdfjsbot · 2013-07-31T20:46:12Z

From: Bot.io (Linux)

Received

Command cmd_makeref from @yurydelendik received. Current queue size: 0

Live output at: http://107.21.233.14:8877/78aca2dc935bfcf/output.txt

pdfjsbot · 2013-07-31T20:46:12Z

From: Bot.io (Windows)

Received

Command cmd_makeref from @yurydelendik received. Current queue size: 0

Live output at: http://107.22.172.223:8877/3f28f53a05ed57b/output.txt

Incrementally render by sending the operator list by chunks as they're ready.

timvandermeij · 2013-07-31T21:01:56Z

That speed optimization is just... wow. :-D

That large map used to take 22 seconds to be fully rendered, whereas it now takes 7 seconds!

pdfjsbot · 2013-07-31T21:09:27Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/78aca2dc935bfcf/output.txt

Total script time: 23.24 mins

Lint: Passed
Make references: Passed
Check references: Passed

pdfjsbot · 2013-07-31T21:14:56Z

From: Bot.io (Windows)

Success

Full output at http://107.22.172.223:8877/3f28f53a05ed57b/output.txt

Total script time: 28.72 mins

Lint: Ignored
Make references: Passed
Check references: Passed

Rob--W · 2015-08-14T09:11:42Z

web/debugger.js

+      panel.appendChild(content);
+      this.table = table;
+    },
+    updateOperatorList: function updateOperatorList(operatorList) {


This is dead code now, because updateOperatorList is not called anywhere.

Never mind, it is called in display/api.js.

yurydelendik reviewed Jul 31, 2013
View reviewed changes

ghost assigned yurydelendik Jul 31, 2013

Incrementally render by sending the operator list by chunks as they'r…

bf72bc9

…e ready.

yurydelendik added a commit that referenced this pull request Jul 31, 2013

Merge pull request #3461 from brendandahl/incremental-rendering2

8d386a5

Incrementally render by sending the operator list by chunks as they're ready.

yurydelendik merged commit 8d386a5 into mozilla:master Jul 31, 2013

brendandahl mentioned this pull request Jul 31, 2013

[PROTOTYPE] Implements chunked rendering #3164

Closed

timvandermeij mentioned this pull request Aug 3, 2013

Begin drawing pages while still downloading data? #1375

Closed

This was referenced Aug 5, 2013

Page becomes empty during zoom #3545

Closed

PDF.js slow at rendering complex image #2618

Closed

Page becomes empty during zoom - another regression from #3461 #3550

Closed

Rob--W reviewed Aug 14, 2015
View reviewed changes

mustafa0x mentioned this pull request Mar 6, 2018

Full PDF doc loaded before single page could be rendered #9537

Closed

Snuffleupagus mentioned this pull request Jun 10, 2019

Investigate how to continue the streams API work #10023

Closed

Incrementally render by sending the operator list by chunks as they're ready. #3461

Incrementally render by sending the operator list by chunks as they're ready. #3461

Conversation

brendandahl commented Jul 10, 2013

brendandahl commented Jul 10, 2013

pdfjsbot commented Jul 10, 2013

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 10, 2013

From: Bot.io (Linux)

Received

pdfjsbot commented Jul 10, 2013

From: Bot.io (Linux)

Failed

pdfjsbot commented Jul 11, 2013

From: Bot.io (Windows)

Failed

Snuffleupagus commented Jul 11, 2013

brendandahl commented Jul 12, 2013

pdfjsbot commented Jul 12, 2013

From: Bot.io (Linux)

Received

pdfjsbot commented Jul 12, 2013

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 12, 2013

From: Bot.io (Linux)

Failed

pdfjsbot commented Jul 12, 2013

From: Bot.io (Windows)

Failed

brendandahl commented Jul 16, 2013

timvandermeij commented Jul 18, 2013

brendandahl commented Jul 18, 2013

pdfjsbot commented Jul 18, 2013

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 18, 2013

From: Bot.io (Linux)

Received

pdfjsbot commented Jul 18, 2013

From: Bot.io (Linux)

Failed

pdfjsbot commented Jul 18, 2013

From: Bot.io (Windows)

Failed

yurydelendik Jul 31, 2013

Choose a reason for hiding this comment

yurydelendik commented Jul 31, 2013

pdfjsbot commented Jul 31, 2013

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 31, 2013

From: Bot.io (Windows)

Success

Published

yurydelendik commented Jul 31, 2013

pdfjsbot commented Jul 31, 2013

From: Bot.io (Linux)

Received

pdfjsbot commented Jul 31, 2013

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 31, 2013

From: Bot.io (Linux)

Failed

pdfjsbot commented Jul 31, 2013

From: Bot.io (Windows)

Failed

yurydelendik commented Jul 31, 2013

pdfjsbot commented Jul 31, 2013

From: Bot.io (Linux)

Received

pdfjsbot commented Jul 31, 2013

From: Bot.io (Windows)

Received

timvandermeij commented Jul 31, 2013

pdfjsbot commented Jul 31, 2013

From: Bot.io (Linux)

Success