feature(collator): queue diffs tail #480

drmick · 2024-11-29T10:35:06Z

No description provided.

codecov · 2024-11-29T10:48:28Z

Codecov Report

Attention: Patch coverage is 48.74275% with 265 lines in your changes missing coverage. Please review.

Project coverage is 44.22%. Comparing base (62730ec) to head (5a1f853).
Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
collator/src/collator/do_collate/finalize.rs	0.00%	52 Missing ⚠️
collator/src/collator/do_collate/mod.rs	57.28%	44 Missing ⚠️
core/src/block_strider/subscriber/mod.rs	0.00%	41 Missing ⚠️
storage/src/store/internal_queue/mod.rs	66.66%	30 Missing and 2 partials ⚠️
collator/src/manager/blocks_cache.rs	0.00%	21 Missing ⚠️
collator/src/manager/mod.rs	0.00%	16 Missing ⚠️
collator/src/internal_queue/queue.rs	85.55%	6 Missing and 7 partials ⚠️
collator/src/queue_adapter.rs	0.00%	12 Missing ⚠️
block-util/src/queue/queue_diff.rs	41.66%	7 Missing ⚠️
collator/src/types.rs	0.00%	5 Missing ⚠️
... and 11 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #480      +/-   ##
==========================================
- Coverage   44.23%   44.22%   -0.02%     
==========================================
  Files         269      269              
  Lines       47575    47817     +242     
  Branches    47575    47817     +242     
==========================================
+ Hits        21047    21147     +100     
- Misses      25438    25578     +140     
- Partials     1090     1092       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

collator/src/collator/do_collate/finalize.rs

collator/src/collator/types.rs

collator/src/collator/do_collate/finalize.rs

block-util/src/queue/queue_diff.rs

collator/src/queue_adapter.rs

collator/src/collator/do_collate/mod.rs

SmaGMan · 2024-12-02T12:01:47Z

collator/src/internal_queue/queue.rs

-    fn clear_session_state(&self) -> Result<()> {
-        self.diffs.clear();
-        self.session_state.truncate()
+    fn clear_uncommitted_state(&self) -> Result<()> {


We call this method when block mismatched. Then we run sync to applied mc block and apply required queue diff for this mc block. Possibly we can get incorrect diff_tail_len because we keep committed_diffs unchanged. Should check this for all places were clear_uncommitted_state called.

collator/src/manager/blocks_cache.rs

SmaGMan · 2024-12-10T10:30:23Z

core/src/block_strider/subscriber/mod.rs

+            .await?;
+        let tail_len = block.block().out_msg_queue_updates.tail_len as usize;
+
+        min_tail_len = Some(min_tail_len.map_or(tail_len, |min: usize| min.min(tail_len)));


Why do we get the minimal tail length from shards? Should it be the longest tail instead?

SmaGMan · 2024-12-27T19:26:28Z

Will be included into feat/msgs-proc-logic-v3

drmick requested a review from SmaGMan November 29, 2024 10:35

drmick force-pushed the feature/queue-diff-len-calc-ref branch 2 times, most recently from 7a3ec2e to c1b9a26 Compare November 29, 2024 10:41

drmick force-pushed the feature/queue-diff-len-calc-ref branch from c1b9a26 to f520b76 Compare November 29, 2024 10:55

SmaGMan reviewed Nov 29, 2024

View reviewed changes

collator/src/collator/do_collate/finalize.rs Outdated Show resolved Hide resolved

SmaGMan reviewed Nov 29, 2024

View reviewed changes

collator/src/collator/do_collate/finalize.rs Outdated Show resolved Hide resolved

SmaGMan reviewed Nov 29, 2024

View reviewed changes

collator/src/collator/types.rs Outdated Show resolved Hide resolved

SmaGMan reviewed Nov 29, 2024

View reviewed changes

collator/src/collator/do_collate/finalize.rs Outdated Show resolved Hide resolved

Base automatically changed from feature/large-queue-in-master to master November 29, 2024 14:45

drmick force-pushed the feature/queue-diff-len-calc-ref branch 2 times, most recently from 627290e to a888612 Compare November 29, 2024 17:51

drmick requested a review from SmaGMan November 29, 2024 17:51

drmick changed the title ~~Feature/queue diff len calc ref~~ feature(collator): queue tail len Nov 29, 2024

drmick requested a review from Rexagon November 29, 2024 18:04

drmick force-pushed the feature/queue-diff-len-calc-ref branch from 6be90f0 to 61804d6 Compare November 30, 2024 21:20

drmick changed the title ~~feature(collator): queue tail len~~ feature(collator): queue diffs tail Nov 30, 2024

drmick force-pushed the feature/queue-diff-len-calc-ref branch from 5265656 to bde3adb Compare November 30, 2024 22:17