Git集成(双向备份)

[HolgerHuo]

No description provided.

[HolgerHuo]

https://github.com/ertuil/overleaf

[AllanChain]

Summary of the Current Investigations

Git Bridge

In Overleaf, the Git bridge and GitHub sync are not entirely the same functions. The Git bridge allows pushing and pulling from git.overleaf.com/project_id, whereas GitHub sync performs bidirectional synchronization from external hosting platforms.

The previously mentioned implementation by USTC only provides a Git bridge. Their olgitbridge uses web APIs for pushing and pulling, and its server does not need to run on the same machine as Overleaf. However, olgitbridge has certain limitations:

Note that contrary to the cloud version this bridge does not use realtime change operations, any files changed by git will result in a "this file has been changed externally" interruption in the online editor.

Overleaf's own implementation of the Git bridge is also tricky: overleaf#782 (comment)

Third Party Data Store (TPDS)

To circumvent the complexity of the Git bridge, we can refer to the implementation of Dropbox synchronization. In Overleaf's codebase, services like Dropbox are referred to as third-party data stores.

We can ignore the fact that the external Gitea server is a Git server and treat it simply as a regular file service:

• POST /repos/{owner}/{repo}/contents can be used to modify files in Gitea.
• Use webhooks to listen for external changes to the repository.

This way, we can heavily reuse the logic of Dropbox synchronization in Overleaf. Good news is that the open-source version has not completely removed the relevant logic for Dropbox, which significantly reduces the workload.

The logic for synchronizing changes from Overleaf to external storage is roughly as follows:

graph LR
    A[Frontend] -->|Document modification| B[Web Service]
    B -->|"/enqueue/web_to_tpds_http_requests"| C[TPDS Service]
    C -->|"Some logic"| D[External data store]

Loading

From modifying a document in the frontend to the web service issuing a /enqueue/web_to_tpds_http_requests request, the logical flow is largely intact. The part responsible for triggering /enqueue/web_to_tpds_http_requests is handled by TpdsUpdateSender.js. The TPDS Service is located in the services/third-party-datastore directory and needs to be implemented.

The logic for synchronizing changes from external storage to Overleaf is roughly as follows:

graph LR
    A[External data store] -->|webhook| B[Web Service]

Loading

The main logic is found in TpdsUpdateHandler.js. Specifically, modifications might be needed in methods such as TpdsController.updateProjectContents to adapt to Gitea's webhook.

[HolgerHuo]

Reference:

• Gitea API for modifying repo
• Gitea can send webhooks for push events to these services, which we can apparently make use of.

This may not be portable to other potential users of this instance, i.e., to integrate other git implementations, anyhow, this is yet to be discussed.

[AllanChain]

Even if we choose to use Git bridge-based solutions, we still have to write different logics to support different Git hosting platforms, because they can have different web hooks, authentication methods, etc.

[Musicminion]

帅的啊 居然真的还有这么多人在为了实现git bridge努力（

从overleaf issue过来的hhh

[leavelet]

From my perspective, git bridge should be considered as a 'version control tool', rather than a 'history recorder'. Users shall trigger commit or tag actions by hand. In such cases, the frequency of commits will be reduced so we can implement them using less sophisticated methods.

I will propose some use cases for the git bridge here:

• Version control and release management
• Automated publishing: Users can integrate Overleaf with external CI systems to publish their work upon release. We can provide GitHub CI templates to facilitate this process
• Non-realtime collaboration: Team members can work on papers sequentially

4. Semi-realtime collaboration: External collaborators can write code locally and push changes to Overleaf users

Note: Real-time, line-based collaborative editing should always be confined to the platform itself, not handled through the git bridge

For implementation simplicity, we propose the following approach:

1. Repository binding: Users can configure a remote repository address and access token with write permissions to enable synchronization between Overleaf and the git repository via git protocol
2. Platform integration: Implement a "Connect Git Account" feature to streamline the token authorization process. Note: git repo integration will be platform agnostic
3. UI enhancement: Add commit and tag functionality to the web interface with appropriate keyboard shortcuts
4. Remote change management: Monitor remote repository changes either through periodic polling (e.g., every 5 minutes) or webhooks. Note: Users may need to resolve conflicts manually using provided web-based tools
5. Feature integration:
   • Incorporate git version comparison capabilities into the existing history comparison tool
   • Implement a viewer to display document changes since the last web-based edit

The user experience and workflow design require further discussion, particularly regarding the handling of rebasing operations and conflict resolution, which present significant challenges.