Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix(agent): Fix buffer directory config and document #15661

Merged
merged 3 commits into from
Jul 30, 2024

Conversation

DStrand1
Copy link
Member

Summary

Fixes buffer directory agent level config and documents the option.

Checklist

  • No AI generated code was used in this PR

Related issues

resolves #15656

@telegraf-tiger telegraf-tiger bot added the chore label Jul 24, 2024
@DStrand1 DStrand1 added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Jul 24, 2024
@josephbrosef
Copy link

hey, i tested the new build and it doesn't seem to resolve the issue. setting buffer_strategy = "disk" and buffer_directory = "/buffer_storage" at the [agent] level does not actually change the buffer strategy. It still uses memory buffer. pretty sure config.BufferStrategy and config.BufferDirectory are always an empty string, so it will always return with a 'memory' buffer.

b, err := NewBuffer(config.Name, config.Alias, bufferLimit, config.BufferStrategy, config.BufferDirectory)

func NewBuffer(name string, alias string, capacity int, strategy string, path string) (Buffer, error) {
registerGob()
bs := NewBufferStats(name, alias, capacity)
switch strategy {
case "", "memory":
return NewMemoryBuffer(capacity, bs)
case "disk":
return NewDiskBuffer(name, path, bs)
}
return nil, fmt.Errorf("invalid buffer strategy %q", strategy)
}

Those 2 values appear to come from the OutputConfig struct (which is why i thought maybe it could be set at the Output level in the referenced bug), but i can not see how those 2 struct values are supposed to get populated with the buffer_strategy and buffer_directory from the [agent] config?

type OutputConfig struct {
Name string
Alias string
ID string
StartupErrorBehavior string
Filter Filter
FlushInterval time.Duration
FlushJitter time.Duration
MetricBufferLimit int
MetricBatchSize int
NameOverride string
NamePrefix string
NameSuffix string
BufferStrategy string
BufferDirectory string
}

Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job @DStrand1! Only one minor comment from my side.

@telegraf-tiger
Copy link
Contributor

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

🥳 This pull request decreases the Telegraf binary size by -3.96 % for linux amd64 (new size: 252.1 MB, nightly size 262.5 MB)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB RPM TAR GZ ZIP
amd64.deb aarch64.rpm darwin_amd64.tar.gz windows_amd64.zip
arm64.deb armel.rpm darwin_arm64.tar.gz windows_arm64.zip
armel.deb armv6hl.rpm freebsd_amd64.tar.gz windows_i386.zip
armhf.deb i386.rpm freebsd_armv7.tar.gz
i386.deb ppc64le.rpm freebsd_i386.tar.gz
mips.deb riscv64.rpm linux_amd64.tar.gz
mipsel.deb s390x.rpm linux_arm64.tar.gz
ppc64el.deb x86_64.rpm linux_armel.tar.gz
riscv64.deb linux_armhf.tar.gz
s390x.deb linux_i386.tar.gz
linux_mips.tar.gz
linux_mipsel.tar.gz
linux_ppc64le.tar.gz
linux_riscv64.tar.gz
linux_s390x.tar.gz

Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update!

Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @DStrand1!

@srebhan srebhan changed the title chore(agent): Fix buffer directory config and document fix(agent): Fix buffer directory config and document Jul 30, 2024
@telegraf-tiger telegraf-tiger bot added the fix pr to fix corresponding bug label Jul 30, 2024
@srebhan srebhan added fix pr to fix corresponding bug and removed fix pr to fix corresponding bug chore labels Jul 30, 2024
@srebhan srebhan merged commit 790c21e into influxdata:master Jul 30, 2024
29 checks passed
@github-actions github-actions bot added this to the v1.31.3 milestone Jul 30, 2024
@DStrand1 DStrand1 deleted the buffer-directory-fix branch October 11, 2024 19:25
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
area/agent fix pr to fix corresponding bug ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Disk Buffer Strategy configuration not functioning/being ignored
4 participants