Skip to content

Commit

Permalink
fix: Prevent crash due to concurrent map access in Go plugin log proc…
Browse files Browse the repository at this point in the history
…essing (#1284)

This commit resolves a critical concurrency issue in the Go plugin log processing that could lead to a crash. The problem was identified as a "concurrent map read and map write" error stemming from shared context map usage without proper isolation. During high log volume situations, the concurrent modification of the context map in the ReceiveLogGroup function and simultaneous access in the Add function caused the crash. The fix introduces a shallow copy of the context map for each log instance, ensuring that each processor works with an independent map and preventing the concurrent access conflicts that could lead to runtime crashes.
  • Loading branch information
quzard committed Dec 29, 2023
1 parent 8b89fa2 commit 6ba9dce
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions pluginmanager/plugin_runner_v1.go
Original file line number Diff line number Diff line change
Expand Up @@ -432,18 +432,26 @@ func (p *pluginv1Runner) ReceiveRawLog(log *pipeline.LogWithContext) {
}

func (p *pluginv1Runner) ReceiveLogGroup(logGroup pipeline.LogGroupWithContext) {
context := logGroup.Context
topic := logGroup.LogGroup.GetTopic()
context[ctxKeyTopic] = topic
for _, log := range logGroup.LogGroup.Logs {
if len(topic) > 0 {
log.Contents = append(log.Contents, &protocol.Log_Content{Key: tagKeyLogTopic, Value: topic})
}
// When UsingOldContentTag is set to false, the tag is now put into the context during cgo.
if !p.LogstoreConfig.GlobalConfig.UsingOldContentTag {
context := map[string]interface{}{}
for key, value := range logGroup.Context {
context[key] = value
}
context[ctxKeyTopic] = topic
context[ctxKeyTags] = logGroup.LogGroup.LogTags
p.ReceiveRawLog(&pipeline.LogWithContext{Log: log, Context: context})
} else {
context := map[string]interface{}{}
for key, value := range logGroup.Context {
context[key] = value
}
context[ctxKeyTopic] = topic
for _, tag := range logGroup.LogGroup.LogTags {
log.Contents = append(log.Contents, &protocol.Log_Content{
Key: tagPrefix + tag.GetKey(),
Expand Down

0 comments on commit 6ba9dce

Please # to comment.