Identify alternatives to automerge #19
Comments
adzialocha
changed the title
adzialocha
added
backend
|
Observation 1: Finding out how to get to the actual applied "deltas" (not encoded etc.) is not too easy in the current Yrs documentation. I could find a test in the code-base though after some time which shows the needed behaviour: let doc = Doc::new();
let text = doc.get_or_insert_text(TEXT_CONTAINER_ID);
let subscription = text.observe(|txn, text_event| {
println!("{:?}", text_event.delta(txn));
});
text.insert(&mut doc.transact_mut(), 0, "hello, world!");Getting to this point was easier with Loro as it's the default API for receiving all updates: let doc = LoroDoc::new();
let text = doc.get_text(TEXT_CONTAINER_ID);
let subscription = doc.subscribe(
&text.id(),
Arc::new(move |event| {
println!("{:?}", event);
}),
);
text.insert(0, "hello, world")?' |
|
Observation 2: Both Yrs and Loro seemed to have adopted the QuillJS "Delta" format to describe updates on the text. I assume that's because these libraries are JavaScript focussed and want to support Quill? For us this format doesn't work as we need the absolute position to update our TextBuffer. It's easy though to convert from quill deltas to absolute numbers: fn quill_delta_to_update(diff_event: DiffEvent<'_>) -> DocumentUpdate {
let mut deltas = Vec::new();
let mut index = 0;
for event in diff_event.events {
match event.diff {
Diff::Text(quill_deltas) => {
for quill in quill_deltas {
let delta = match quill {
TextDelta::Retain { retain, .. } => {
index += retain;
continue;
}
TextDelta::Insert { insert, .. } => Delta::Insert {
index,
chunk: insert,
},
TextDelta::Delete { delete } => Delta::Remove { index, len: delete },
};
deltas.push(delta);
}
}
_ => continue,
}
}
// ...
} |
|
Observation 3: Both Yrs and Loro use the callback-pattern to subscribe to document changes. That seems fine and might come again from an JavaScript focussed design (?). To bridge this with an "async" GTK world we need to use a blocking send for sending updates via an channel to the frontend. If this blocking send is running within the same thread as the async runtime we might run into trouble. Tokio actually panics if you try to do that with an tokio channel implementation, using a different implementation, for example let doc = LoroDoc::new();
let text = doc.get_text(TEXT_CONTAINER_ID);
let (update_tx, update_rx) = async_channel::bounded::<DocumentUpdate>(64);
let subscription = {
let update_tx = update_tx.clone();
doc.subscribe(
&text.id(),
Arc::new(move |event| {
let update = quill_delta_to_update(event);
let _ = update_tx.send_blocking(update);
}),
)
};We could run all CRDT logic in a separate thread to avoid this issue, but I assume there's then another cost through multi-threading. |
|
Observation 4: Yrs examples on their main documentation page assume that we know the state (vector clock) of the other peer to encode the delta. let update = doc.transact().encode_diff_v1(&StateVector::decode_v1(&remote_timestamp).unwrap());This makes sense in a setting where every peer is connected to every other ("fully connected graph") but doesn't scale well and on top doesn't resemble the design we're following (gossip overlay, intermediary "caching" nodes, broadcast-only transports like BLE). It took a while to find a way to calculate the delta without knowledge of another peer. |
It's still unclear if we really want to switch, but there's some worries about tying ourselves too close to Automerge.
Reasons to not stick to Automerge:
Alternatives so far:
Looking at this list now after digging a bit more into Yrs and Loro it still seems that they also suffer from 1., but 3. is the point where they shine!
The text was updated successfully, but these errors were encountered: