Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Sync active request byrange ids logs #6914

Merged
merged 4 commits into from
Feb 10, 2025

Conversation

dapplion
Copy link
Collaborator

@dapplion dapplion commented Feb 5, 2025

Issue Addressed

Writing and running tests I noted that the sync RPC requests are very verbose now.

DataColumnsByRootRequestId { id: 123, requester: Custody(CustodyId { requester: CustodyRequester(SingleLookupReqId { req_id: 121, lookup_id: 101 }) }) }

Since this Id is logged rather often I believe there's value in

  1. Making them more succinct for log verbosity
  2. Make them a string that's easy to copy and work with elastic

Proposed Changes

Write custom Display implementations to render Ids in a more DX format

_ DataColumnsByRootRequestId with a block lookup_

123/Custody/121/Lookup/101

DataColumnsByRangeRequestId

123/122/RangeSync/0/5492900659401505034

Also made the logs format and text consistent across all methods

@dapplion dapplion requested a review from jxs as a code owner February 5, 2025 15:55
@jimmygchen jimmygchen added ready-for-review The code is ready for review syncing UX-and-logs labels Feb 6, 2025
// not losing information
impl_display!(BlocksByRangeRequestId, "{}/{}", id, parent_request_id);
impl_display!(BlobsByRangeRequestId, "{}/{}", id, parent_request_id);
impl_display!(DataColumnsByRangeRequestId, "{}/{}", id, parent_request_id);
Copy link
Member

@jimmygchen jimmygchen Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely better than before! although it may still be not as straight forward to search in an indexed log viewer, compared to other logs where we have key/values.

e.g.

123/122/RangeSync/0/5492900659401505034

This is quite hard to read without memorizing the order or looking at the code.

Also if we do want to go ahead with this format, would it make more sense to have the order the other way around? i.e. parent request id first? (actually i realise it doesn't matter coz we have requester at the end)

},
},
};
assert_eq!(format!("{id}"), "123/122/RangeSync/0/54");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found having consecutive numbers make it harder to guess what they are.

Would something like this be easier to understand?
123/Parent/122/RangeSync/Batch/0/Chain/54

Copy link
Collaborator Author

@dapplion dapplion Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are debug logs, only use will use this. Have you seen @AgeManning 's batch state codes? It's even harder to follow 🤣 . Remember that before we just had a number with no information at all. This IDs get logged quite a lot when sync is running I believe succinctness is good

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree with this, because for anyone else to read this (even anyone from our team) we'd have to look it up in the code to know how to use this log, and I don't think adding Parent and Batch to it makes it a lot more verbose than it is now.

Copy link
Member

@jimmygchen jimmygchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added my view on the new log format, no strong opinion though if this makes it easier for you to debug.

@jimmygchen jimmygchen added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Feb 6, 2025
Copy link
Member

@jimmygchen jimmygchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see comment above.
#6914 (comment)

If it helps with debugging happy to merge now and revisit later

@michaelsproul michaelsproul added v7.0.0-beta.0 New release c. Q1 2025 ready-for-merge This PR is ready to merge. and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Feb 10, 2025
mergify bot added a commit that referenced this pull request Feb 10, 2025
michaelsproul added a commit that referenced this pull request Feb 10, 2025
Squashed commit of the following:

commit cbc378a
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Feb 5 12:55:13 2025 -0300

    De-duplicate code

commit 4583116
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Jan 26 14:16:55 2025 +0400

    Add log when sending batch to processor

commit cacfa06
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Jan 26 13:01:02 2025 +0400

    Concise request Ids for sync

commit 4627f38
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Sun Jan 26 12:00:03 2025 +0400

    Consistent logging in sync rpc requests
@michaelsproul michaelsproul mentioned this pull request Feb 10, 2025
7 tasks
@mergify mergify bot merged commit f35213e into sigp:unstable Feb 10, 2025
31 checks passed
@dapplion dapplion deleted the sync-active-request-byrange-ids-logs branch February 10, 2025 18:52
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
ready-for-merge This PR is ready to merge. syncing UX-and-logs v7.0.0-beta.0 New release c. Q1 2025
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants