Ingest dev guide #1415
Ingest dev guide #1415
Conversation
|
Preview is available here: |
docs/data/ingest-library/developer_guide/ledgerbackends/README.mdx
Outdated
Show resolved
Hide resolved
docs/data/ingest-library/developer_guide/ledgerbackends/README.mdx
Outdated
Show resolved
Hide resolved
| sidebar_position: 10 | ||
| --- | ||
|
|
||
| A ledger backend is a source of Stellar network ledger data. he ingest library supports two primary ledger backends, both implementing the [ledgerbackend](https://github.com/stellar/go/blob/master/ingest/ledgerbackend/ledger_backend.go) interface. This interface always returns ledger data as `xdr.LedgerCloseMeta`, which is XDR-encoded ledger metadata object. |
There was a problem hiding this comment.
It would be helpful to mention a summary of the different types of data that are available in xdr.LedgerCloseMeta
There was a problem hiding this comment.
docs/data/ingest-library/developer_guide/ledgerbackends/README.mdx
Outdated
Show resolved
Hide resolved
docs/data/ingest-library/developer_guide/ledgerbackends/captivecore.mdx
Outdated
Show resolved
Hide resolved
|
This should have been a comment on the nav conflict rebase - #1405 - but I missed that review and I'm just seeing it here. We wanted "ingest library/SDK" to be under the "Indexers -> Build your own" not in the top-level nav. Not sure if you want to handle that here since you're mostly just adding actual content. If not, we should address that on the review/merge of the feature branch back into master |
docs/data/ingest-library/developer_guide/ledgerbackends/bufferedstoragebackend.mdx
Outdated
Show resolved
Hide resolved
|
Preview is available here: |
4 similar comments
|
Preview is available here: |
|
Preview is available here: |
|
Preview is available here: |
|
Preview is available here: |
|
|
||
| ## Package Structure | ||
|
|
||
| The ingest library is broadly split into three main packages: |
There was a problem hiding this comment.
Should we include mention of HistoryArchive usage via the github.com/stellar/go/historyarchive package at this level it's significant for ledger entry aspect. And the xdr bindings from github.com/stellar/go/xdr as it contains all data models.
I think it might be worthwhile to include the formal package names for each item mentioned either expressed as code formatted markup likegithub.heygears.com/stellar/go/ingest/ledgerbackend or as link to gh repo path github.com/stellar/go/ingest/ledgerbackend
There was a problem hiding this comment.
I think it might be worthwhile to include the formal package names for each item mentioned either expressed as code formatted markup likegithub.heygears.com/stellar/go/ingest/ledgerbackend or as link to gh repo path github.com/stellar/go/ingest/ledgerbackend
done.
There was a problem hiding this comment.
Should we include mention of HistoryArchive usage via the github.com/stellar/go/historyarchive package at this level it's significant for ledger entry aspect. And the xdr bindings from github.com/stellar/go/xdr as it contains all data models.
I'm not sure about including historyarchives here since it's outside the ingest package at least for now. For xdr, I'll add more details about the data available in ledgerclosemeta object (see Tamir's comment here) and will include the link to the xdr package there.
There was a problem hiding this comment.
Yes, historyarchive is not under github.com/stellar/go/ingest, but its functionality is a subset of ingestion development, we reference consuming history archives to get ledger entries as a use case for ingest sdk in the introduction
docs/data/ingest-library/developer_guide/ledgerbackends/captivecore.mdx
Outdated
Show resolved
Hide resolved
I'll fix the nav for ingest library on #1346. [edit] done on #1422 |
|
Preview is available here: |
Co-authored-by: tamirms <tamirms@gmail.com>
Co-authored-by: tamirms <tamirms@gmail.com>
….mdx Co-authored-by: tamirms <tamirms@gmail.com>
….mdx Co-authored-by: tamirms <tamirms@gmail.com>
urvisavla
force-pushed
the
ingest-dev-guide
branch
from
05458d7 to
bb6ae15
Compare
|
Preview is available here: |
content looks solid, do you think it's worthwhile to carve out the processors and ledgerreaders sections to second pr which could be merged directly to main later? We could proceed to merge this and I think we're then ready to merge ingest feature branch to main. I'm ready to approve, thanks! |
|
Preview is available here: |
Sounds good. I've made a few updates including adding architecture diagrams, section on the LedgerCloseMeta struct and content to the LedgerReaders page. If everything looks good, I can go ahead and merge it. The only remaining items are adding information about Processors and the history archives. |
static/assets/ingest-sdk/bufferedstoragebackend_architecture.png
Outdated
Show resolved
Hide resolved
docs/data/indexers/build-your-own/ingest-sdk/developer_guide/architecture.mdx
Outdated
Show resolved
Hide resolved
…rchitecture.mdx Co-authored-by: shawn <sreuland@users.noreply.github.com>
docs/data/indexers/build-your-own/ingest-sdk/developer_guide/ledgerbackends/captivecore.mdx
Outdated
Show resolved
Hide resolved
docs/data/indexers/build-your-own/ingest-sdk/developer_guide/ledgerbackends/captivecore.mdx
Outdated
Show resolved
Hide resolved
docs/data/indexers/build-your-own/ingest-sdk/developer_guide/ledgerprocessors.mdx
Outdated
Show resolved
Hide resolved
…edgerbackends/captivecore.mdx Co-authored-by: shawn <sreuland@users.noreply.github.com>
…edgerbackends/captivecore.mdx Co-authored-by: shawn <sreuland@users.noreply.github.com>
|
Preview is available here: |
3 similar comments
|
Preview is available here: |
|
Preview is available here: |
|
Preview is available here: |
docs/data/indexers/build-your-own/ingest-sdk/developer_guide/ledgerbackends/README.mdx
Outdated
Show resolved
Hide resolved
|
Preview is available here: |
urvisavla
merged commit 1779897
into
stellar:feature-branch/platform-docpoc25
addresses #1328
This PR addressed the ledger backend documentation for the ingest library, as outlined in
requirements. It covers prerequisites, installation, setup, and code examples for ledger backends. Placeholders have been added for processors and ledger readers. I'd like to get some initial feedback on the structure and level of details before expanding remaining sections.