Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

truncate single file upon creation #1143

Merged
merged 1 commit into from
Feb 20, 2025
Merged

Conversation

HNOONa-0
Copy link
Contributor

A small change that writes logs to beginning of the file when its size exceeds max_log_size constraint.

Fixes: #1089

Copy link

google-cla bot commented Dec 16, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@HNOONa-0
Copy link
Contributor Author

HNOONa-0 commented Dec 16, 2024

Hello @sergiud, I hope you had a pleasant vacation. PTAL!

@sergiud
Copy link
Collaborator

sergiud commented Jan 19, 2025

Thanks for the PR and apologies for the late reply.

I'm not sure I understand the change. The log output file is truncated unconditionally; no check against max_log_size is performed. Can you elaborate why this still fixes the issue?

@HNOONa-0
Copy link
Contributor Author

HNOONa-0 commented Jan 19, 2025

Thanks for the PR and apologies for the late reply.

I'm not sure I understand the change. The log output file is truncated unconditionally; no check against max_log_size is performed. Can you elaborate why this still fixes the issue?

There are two cases:
If FLAGS_timestamp_in_logfile_name is true, then a new file with a timestamp is created.

Else, it's implied that we are writing to a single log file, and in this case, whenever we are attempting to create a new file, it's because:

  1. The log file is created for the first time.
  2. The log file is being recreated due to length overflow in file size in this part of implementation:

    glog/src/logging.cc

    Lines 1110 to 1117 in 6c5c692

    if (file_length_ >> 20U >= MaxLogSize() || PidHasChanged()) {
    file_ = nullptr;
    file_length_ = bytes_since_flush_ = dropped_mem_length_ = 0;
    rollover_attempt_ = kRolloverAttemptFrequency - 1;
    }
    // If there's no destination file, make one before outputting
    if (file_ == nullptr) {

When file_ is null, the logger will attempt to create a new logging file. In both these scenarios, the file should be truncated to reset the file size, hence the O_TRUNC flag. Is my understanding correct so far?

@sergiud
Copy link
Collaborator

sergiud commented Jan 19, 2025

I see, thanks. Could you please extend the comment to mention that the log is recreated in the else branch due to size overflow as per 2?

@HNOONa-0
Copy link
Contributor Author

Hey @sergiud!
I’ve extended the comments as you suggested and added a new test method to validate these changes. In this test, I turned off the timestamp flag and constrained the log_size to 1MB. Then, I logged 2e4 lines. Although the total lines exceed the 1MB threshold, the test ensures that the file size remains within the 1MB limit.

@codecov-commenter
Copy link

codecov-commenter commented Feb 18, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.29%. Comparing base (7fcf58a) to head (d693e22).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1143      +/-   ##
==========================================
+ Coverage   62.14%   62.29%   +0.15%     
==========================================
  Files          19       19              
  Lines        2327     2331       +4     
  Branches      846      834      -12     
==========================================
+ Hits         1446     1452       +6     
  Misses        599      599              
+ Partials      282      280       -2     
Files with missing lines Coverage Δ
src/logging.cc 69.47% <100.00%> (+0.27%) ⬆️

@HNOONa-0
Copy link
Contributor Author

HNOONa-0 commented Feb 20, 2025

Hey @sergiud, sorry for the delay.
I've made a few changes to allow appending when the file isn't at max size, as it was causing some other tests to fail. I also made some minor refactors in the test. The tests should all pass hopefully.

@sergiud
Copy link
Collaborator

sergiud commented Feb 20, 2025

Thanks!

@sergiud sergiud merged commit 4f007d9 into google:master Feb 20, 2025
132 of 136 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Apply max_log_file constraint when timestamp_in_log_file=false
3 participants