Helm is a package management tool that uses Charts to deploy pre-configured sets of Kubernetes resources. The Logzio-fluentd chart enables you to transmit logs from your Kubernetes cluster to Logz.io via Fluentd. Fluentd's flexibility and availability of plugins allow for easy distribution of logs to various third-party services, including Logz.io.
The chart defaults to configuration for Conatinerd CRI. If your cluster uses Docker as CRI, please refer to daemonset.containerdRuntime
in the configuration table.
Note: This chart is for shipping logs only. For a chart that ships all telemetry data, including logs, metrics, traces, and SPM, use our Logzio Monitoring chart.
Your DaemonSet will be deployed under the namespace monitoring
.
kubectl create namespace monitoring
helm repo add logzio-helm https://logzio.github.io/logzio-helm
The following command installs the Chart using the default values. If you want to modify any settings, append the --set
flag(s) to this command along with the parameters you wish to change. For more information and examples, refer to the configuration table.
You can find additional details on how to customize the Chart's values here.
Replace <<LOG-SHIPPING-TOKEN>>
with the token of the account you want to ship to.
Replace <<LISTENER-HOST>>
with your account's listener host. You can find your listener in your manage tokens page.
helm install -n monitoring \
--set secrets.logzioShippingToken='<<LOG-SHIPPING-TOKEN>>' \
--set secrets.logzioListener='<<LISTENER-HOST>>' \
logzio-fluentd logzio-helm/logzio-fluentd
If adding the secret directly through the Helm chart is not possible or secure (for example, due to the absence of secure value-file storage), you can override the secret's name by modifying secretName
in the values file. This adjustment allows you to place an external secret containing the keys logzioShippingToken
and logzioListener
through alternative methods.
Give your logs some time to get from your system to ours, and then open Logz.io.
This table contains all the parameters in values.yaml
. If you wish to change the default values, specify each parameter using the --set key=value
argument to helm install
in step 2. For example:
helm install -n monitoring \
--set terminationGracePeriodSeconds=40 \
--set daemonset.logzioLogLevel=debug \
--set-file configmap.extraConfig=/path/to/config.yaml \
logzio-fluentd logzio-helm/logzio-fluentd
Parameter | Description | Default |
---|---|---|
image |
The logzio-fluentd docker image. | logzio/logzio-fluentd |
imageTag |
The logzio-fluentd docker image tag. | 1.4.0 |
nameOverride |
Overrides the Chart name for resources. | "" |
fullnameOverride |
Overrides the full name of the resources. | "" |
apiVersions.daemonset |
Daemonset API version. | apps/v1 |
apiVersions.serviceAccount |
Service Account API version. | v1 |
apiVersions.clusterRole |
Cluster Role API version. | rbac.authorization.k8s.io/v1 |
apiVersions.clusterRoleBinding |
Cluster Role Binding API version. | rbac.authorization.k8s.io/v1 |
apiVersions.configmap |
Configmap API version. | v1 |
apiVersions.secret |
Secret API version. | v1 |
namespace |
Chart's namespace. | monitoring |
fargateLogRouter.enabled |
Boolen to decide if to configure fargate log router | false |
env_id |
Add to your logs field env_id with identification of the environment you're shipping logs from. |
"" |
isRBAC |
Specifies whether the Chart should be compatible to a RBAC cluster. If you're running on a non-RBAC cluster, set to false . |
true |
isPrivileged |
Specifies whether to run the Damonset with priviliged security context | false |
serviceAccount.name |
Name of the service account. | "" |
daemonset.podSecurityContext |
Security context for the pod level | {} |
daemonset.securityContext |
Security context for the container level | {} |
daemonset.initContainerSecurityContext |
Security context for the init container | {} |
daemonset.tolerations |
Set tolerations for all DaemonSet pods. | See values.yaml. |
daemonset.nodeSelector |
Set nodeSelector for all DaemonSet pods. | {"kubernetes.io/os": "linux"} |
daemonset.affinity |
Set affinity rules for the scheduler to determine where all DaemonSet pods can be placed. | |
daemonset.fluentdSystemdConf |
Controls whether Fluentd system messages will be enabled. | disable |
daemonset.fluentdPrometheusConf |
Controls the launch of a prometheus plugin that monitors Fluentd. | false |
daemonset.includeNamespace |
Use if you wish to send logs from specific k8s namespaces, space delimited. Should be in the following format: kubernetes.var.log.containers.**_<<NAMESPACE-TO-INCLUDE>>_** kubernetes.var.log.containers.**_<<ANOTHER-NAMESPACE>>_** . |
"" |
daemonset.kubernetesVerifySsl |
Enables to validate SSL certificates. | true |
daemonset.auditLogFormat |
Match Fluentd's format for kube-apiserver audit logs. Set to audit-json if your audit logs are in json format. |
audit |
daemonset.containerdRuntime |
Deprecated from chart version 0.1.0. Determines whether to use a configuration for a Containerd runtime. Set to false if your cluster doesn't use Containerd as CRI. |
true |
daemonset.cri |
Container runtime interface of the cluster. Used to determine which configuration to use when concatenating partial logs. Valid options are: docker , containerd . |
containerd |
daemonset.LogFileRefreshInterval |
The interval of refreshing the list of watch file for log files.. | 60s |
daemonset.logzioBufferType |
Specifies which plugin to use as the backend. | file |
daemonset.logzioBufferPath |
Path of the buffer. | /var/log/fluentd-buffers/stackdriver.buffer |
daemonset.logzioOverflowAction |
Controls the behavior when the queue becomes full. | block |
daemonset.logzioChunkLimitSize |
Maximum size of a chunk allowed. | 2M |
daemonset.logzioQueueLimitLength |
Maximum length of the output queue. | 6 |
daemonset.logzioFlushInterval |
Interval, in seconds, to wait before invoking the next buffer flush. | 5s |
daemonset.logzioRetryMaxInterval |
Maximum interval, in seconds, to wait between retries. | 30 |
daemonset.logzioRetryForever |
If true, plugin will retry flushing forever | true |
daemonset.logzioFlushThreadCount |
Number of threads to flush the buffer. | 2 |
daemonset.logzioLogLevel |
The log level for this container. | info |
daemonset.excludeFluentdPath |
Path to fluentd logs file, to exclude them from the logs that Fluent tails. | /var/log/containers/*fluentd*.log |
daemonset.extraExclude |
A comma-seperated list (no spaces), of more paths to exclude from the Fluentd source that tails containers logs. For example - /path/one.log,/path/two.log | "" |
daemonset.containersPath |
Path for containers logs. | "/var/log/containers/*.log" |
daemonset.posFile |
Path for containers logs pos file. | "/var/log/fluentd-containers.log.pos" |
daemonset.logType |
Set log type for the logs. | "k8s" |
daemonset.extraEnv |
If needed, more env vars can be added with this field. | [] |
daemonset.resources |
Allows you to set the resources for Fluentd Daemonset. | See values.yaml. |
daemonset.extraVolumeMounts |
If needed, more volume mounts can be added with this field. | [] |
daemonset.terminationGracePeriodSeconds |
Termination period (in seconds) to wait before killing Fluentd pod process on pod shutdown. | 30 |
daemonset.extraVolumes |
If needed, more volumes can be added with this field. | [] |
daemonset.init.extraVolumeMounts |
If needed, more volume mounts to the init container can be added with this field. | [] |
daemonset.init.containerImage |
Init container image for the fluentd daemonset. | busybox |
daemonset.priorityClassName |
Set priorityClassName for all DaemonSet pods. | "" |
daemonset.updateStrategy |
Strategy to use when updating the Daemonset. | {} |
windowsDaemonset.enabled |
Enables Fluentd Daemonset for Windows. | true |
windowsDaemonset.kubernetesVerifySsl |
Enables to validate SSL certificates (windows). | true |
windowsDaemonset.auditLogFormat |
Match Fluentd's format for kube-apiserver audit logs. Set to audit-json if your audit logs are in json format. (windows) |
audit |
windowsDaemonset.containerdRuntime |
Deprecated from chart version 0.1.0. Determines whether to use a configuration for a Containerd runtime. Set to false if your cluster doesn't use Containerd as CRI. (windows) |
true |
windowsDaemonset.cri |
Container runtime interface of the cluster. Used to determine which configuration to use when concatenating partial logs (windows). Valid options are: docker , containerd . |
containerd |
windowsDaemonset.LogFileRefreshInterval |
The interval of refreshing the list of watch file for log files.. | 60s |
windowsDaemonset.logzioBufferType |
Specifies which plugin to use as the backend. | file |
windowsDaemonset.logzioBufferPath |
Path of the buffer. (windows) | /var/log/fluentd-buffers/stackdriver.buffer |
windowsDaemonset.logzioOverflowAction |
Controls the behavior when the queue becomes full. (windows) | block |
windowsDaemonset.logzioChunkLimitSize |
Maximum size of a chunk allowed. (windows) | 2M |
windowsDaemonset.logzioQueueLimitLength |
Maximum length of the output queue. (windows) | 6 |
windowsDaemonset.logzioFlushInterval |
Interval, in seconds, to wait before invoking the next buffer flush. (windows) | 5s |
windowsDaemonset.logzioRetryMaxInterval |
Maximum interval, in seconds, to wait between retries. (windows) | 30 |
windowsDaemonset.logzioRetryForever |
If true, plugin will retry flushing forever (windows) | true |
windowsDaemonset.logzioFlushThreadCount |
Number of threads to flush the buffer. (windows) | 2 |
windowsDaemonset.logzioLogLevel |
The log level for this container. (windows) | info |
windowsDaemonset.excludeFluentdPath |
Path to fluentd logs file, to exclude them from the logs that Fluent tails. | /var/log/containers/*fluentd*.log |
windowsDaemonset.extraExclude |
A comma-seperated list (no spaces), of more paths to exclude from the Fluentd source that tails containers logs. For example - /path/one.log,/path/two.log | "" |
windowsDaemonset.containersPath |
Path for containers logs. | "/var/log/containers/*.log" |
windowsDaemonset.extraEnv |
If needed, more env vars can be added with this field. (windows) | [] |
windowsDaemonset.resources |
Allows you to set the resources for Fluentd Daemonset. (windows) | See values.yaml. |
windowsDaemonset.extraVolumeMounts |
If needed, more volume mounts can be added with this field. (windows) | [] |
daemonset.terminationGracePeriodSeconds |
Termination period (in seconds) to wait before killing Fluentd pod process on pod shutdown. | 30 |
windowsDaemonset.extraVolumes |
If needed, more volumes can be added with this field. (windows) | [] |
windowsDaemonset.priorityClassName |
Set priorityClassName for all DaemonSet pods. (windows) | "" |
windowsDaemonset.updateStrategy |
Strategy to use when updating the Daemonset. | {} |
windowsDaemonset.nodeSelector |
Set nodeSelector for all DaemonSet pods. | {"kubernetes.io/os": "windows"} |
clusterRole.rules |
Configurable cluster role rules that Fluentd uses to access Kubernetes resources. | See values.yaml. |
secrets.logzioShippingToken |
Secret with your logzio shipping token. | "" |
secrets.logzioListener |
Secret with your logzio listener host. listener.logz.io . |
" " |
secrets.customEndpoint |
Secret with your custom endpoint, for example:http://endpoint:8080 . Overrides secrets.logzioListener |
"" |
secrets.enabled |
When true , the logzio secret will be created and managed by this Chart. If you're managing the logzio secret by yourself, set to false . |
true |
secretName |
Name of the secret in case it's placed from an external source. | logzio-logs-secret |
configMapIncludes |
Initial includes for fluent.conf . |
See values.yaml. |
configmap.extraConfig |
If needed, more Fluentd configuration can be added with this field. | {} |
configmap.fluent |
Configuration for fluent.conf . |
See values.yaml. |
configmap.kubernetes |
Configuration for kubernetes.conf . |
See values.yaml. |
configmap.system |
Configuration for system.conf . |
See values.yaml. |
configmap.systemd |
Configuration for systemd.conf . |
See values.yaml. |
configmap.kubernetesContainerd |
Deprecated from chart version 0.1.0. Configuration for kubernetes-containerd.conf . This is the configuration that's being used when daemonset.containerdRuntime is set to true |
See values.yaml. |
configmap.partialDocker |
Configuration for partial-docker.conf . Used to concatenate partial logs that split due to large size, for docker cri. |
See values.yaml. |
configmap.partialContainerd |
Configuration for partial-containerd.conf . Used to concatenate partial logs that split due to large size, for containerd cri. |
See values.yaml. |
configmap.audit |
Configuration for audit.conf . |
See values.yaml. |
configmap.auditJson |
Configuration for audit-json.conf . This is the configuration that's being used when daemonset.auditLogFormat is set to audit-json |
See values.yaml. |
configmap.envId |
Config snippet for adding env_id field to logs |
See values.yaml. |
configmap.customSources |
Add sources to the Fluentd configuration | "" |
configmap.customFilters |
Add filters to the Fluentd configuration | "" |
configmap.customFilterAfter |
Add filters to the Fluentd configuration, after default filters | "" |
logLevelFilter |
Add log level filter. Regex of the log level(s) you want to ship. For example, if you want to ship warning and error logs, use WARNING|ERROR . Possible levels are: DEBUG , INFO , WARNING , ERROR , TRACE . |
"" |
Note: If you're adding your own configuration file via configmap.extraConfig
:
- Add a
--set-file
flag to yourhelm install
command, as seen in the example above. - Make sure that the
yaml
file with your configuration is in the following format:
my-custom-conf-name.conf: |-
# .....
# your config
# .....
my-custom-conf-name2.conf: |-
# .....
# your config
# .....
In some cases (i.e spot clusters) where the pods/nodes are replaced frequently, the pull rate limit for images pulled from dockerhub might be reached, with an error:
You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits
.
In these cases we can use the following --set
command to use an alternative image repository:
--set image=public.ecr.aws/logzio/logzio-fluentd
--set daemonset.init.containerImage=public.ecr.aws/docker/library/busybox
To add a log_type
field with a custom value to each log, you can use the annotation key log_type
with a custom value. The annotation will be automatically parsed into a log_type
field with the provided value.
e.g:
...
metadata:
annotations:
log_type: "my_type"
Will result with the following log (json):
{
...
,"log_type": "my_type"
...
}
The command removes all the k8s components associated with the chart and deletes the release.
To uninstall the logzio-fluentd
deployment:
helm uninstall -n monitoring logzio-fluentd
Fluentd splits multiline logs by default. If your original logs span multiple lines, you may find that they arrive in your Logz.io account split into several partial logs.
The Logz.io Docker image comes with a pre-built Fluentd filter plug-in that can be used to concatenate multiline logs. The plug-in is named fluent-plugin-concat
and you can view the full list of configuration options in the GitHub project.
The following is an example of a multiline log sent from a deployment on a k8s cluster:
2021-02-08 09:37:51,031 - errorLogger - ERROR - Traceback (most recent call last):
File "./code.py", line 25, in my_func
1/0
ZeroDivisionError: division by zero
Fluentd's default configuration will split the above log into 4 logs, 1 for each line of the original log. In other words, each line break (\n
) causes a split.
To avoid this, you can use the fluent-plugin-concat
and customize the configuration to meet your needs. The additional configuration is added to:
kubernetes.conf
for RBAC/non-RBAC DaemonSetkubernetes-containerd.conf
for Containerd DaemonSet
For the above example, we could use the following regex expressions to demarcate the start and end of our example log:
<filter **>
@type concat
key message # The key for part of multiline log
multiline_start_regexp /^[0-9]{4}-[0-9]{2}-[0-9]{2}/ # This regex expression identifies line starts.
</filter>
If you want to ship logs from any of the nodes that have a taint, make sure that the taint key values are listed in your in your daemonset configuration as follows:
tolerations:
- key:
operator:
value:
effect:
To determine if a node uses taints as well as to display the taint keys, run:
kubectl get nodes -o json | jq ".items[]|{name:.metadata.name, taints:.spec.taints}"
If you want to ship logs from pods that are running on fargate set the fargateLogRouter.enabled
value to true, the follwing will deploy a dedicated aws-observability
namespace and a configmap
for fargate log router. More information about eks fargate logging can be found here
helm install \
--set fargateLogRouter.enabled=true \
--set secrets.logzioShippingToken='<<LOG-SHIPPING-TOKEN>>' \
--set secrets.logzioListener='<<LISTENER-HOST>>' \
logzio-fluentd logzio-helm/logzio-fluentd
In order to monitor fluentd and collect input & output metrics. You can
enable prometheus configuration with the daemonset.fluentdPrometheusConf
and windowsDaemonset.fluentdPrometheusConf
parameter (default to false).
When enabling promehteus configuration, the pod collects and exposes fluentd metrics on port 24231
, /metrics
endpoint. The templates contains annotations to easly ship when using promehteus shipper or logzio-telemetry
chart. Monitoring Windows fluentd is not supported.
By default the fluentd image for windows-server supports windows server 2019. If needed, the fluentd image can be changed to support windows server 2022 with the following commands:
--set windowsImage=logzio/fluentd-windows-2022 \
--set windowsImageTag=0.0.5
- 0.30.6:
- Upgrade fluentd version to
1.18.0
- Upgrade fluentd version to
- 0.30.5:
- Upgrade fluentd version to
1.17.1
- Upgrade fluentd version to
- 0.30.4:
- Fix
nodeSelector
indentation
- Fix
- 0.30.3:
- Resolve
nodeSelector
bug
- Resolve
- 0.30.2:
- Remove default resources
limits
- 0.30.1:
- Handle empty etcd
log
key, populated based onmessage
key. - 0.30.0:
- Upgrade fluentd version to
1.16.5
- Fix bug of
env-id.conf
Expand to check old versions
- 0.29.2:
- Enhanced env_id handling to support both numeric and string formats.
- 0.29.1:
- Added
enabled
value, to conditianly control the deployment of this chart by a parent chart. - Added
daemonset.LogFileRefreshInterval
andwindowsDaemonset.LogFileRefreshInterval
values, to control list of watched log files refresh interval.
- Added
- 0.29.0:
- EKS Fargate logging:
- Send logs to port
8070
in logzio listener (instead of port5050
)
- 0.28.1:
- Added
windowsDaemonset.enabled
customization.
- Added
- 0.28.0:
- Added
daemonset.initContainerSecurityContext
customization. - Added
daemonset.updateStrategy
customization.
- Added
- 0.27.0:
- Added
daemonset.podSecurityContext
,daemonset.securityContext
customization.
- Added
- 0.26.0:
- Bump docker image to
1.5.1
. - Add ability to configure pos file for containers logs.
- Bump docker image to
- 0.25.0:
- Add parameter
isPrivileged
to allow running Daemonset with priviliged security context. - Bug fix: Fix template for
fluentd.serviceAccount
, and fix use of template in service account.
- Add parameter
- 0.24.0:
- Add parameter
configmap.customFilterAfter
that allows adding filters AFTER built-in filter configuration. - Added
daemonset.init.containerImage
customization. - Added fluentd image for windows server 2022.
- Add parameter
- 0.23.0:
- Allow filtering logs by log level with
logLevelFilter
.
- Allow filtering logs by log level with
- 0.22.0:
- Add custom endpoint option with
secrets.customEndpoint
.
- Add custom endpoint option with
- 0.21.0:
- Bump docker image to
1.5.0
:- Upgrade fluentd to
1.16
. - Upgrade gem
fluent-plugin-logzio
to0.2.2
:- Do not retry on 400 and 401. For 400 - try to fix log and resend.
- Generate a metric (
logzio_status_codes
) for response codes from Logz.io.
- Upgrade fluentd to
- 0.20.3:
- ezKonnect support: Added
logz.io/application_type
to type annotation check .
- ezKonnect support: Added
- 0.20.2:
- Upgrade docker image
logzio/logzio-fluentd
to1.4.0
:- Use fluentd's retry instead of retry in code (raise exception on non-2xx response).
- Upgrade docker image
- 0.20.1:
- Added log level detection for fargate log router
- Remove
namespace
value, replaced byRealese.namespace
in all templates
- 0.20.0:
- Upgraded windows image to
logzio/windows:0.0.2
:- Added prometheus monitor plugin
- Added dedot plugin
- Updated
windowsDaemonset.fluentdPrometheusConf
- now controls prometheus config for collecting and exposing fluentd metrics.
- Upgraded windows image to
- 0.19.0:
- Upgraded image to
logzio/logzio-fluentd:1.3.1
:- Added prometheus monitor plugin
- Updated
daemonset.fluentdPrometheusConf
- now controls prometheus config for collecting and exposing fluentd metrics.
- Upgraded image to
- 0.18.0:
- Added log_level detection for "warn" level.
- 0.17.0:
- Add
secrets.enabled
to control secret creation and management. (#194)
- Add
- 0.16.0:
- Increased memory request and limit to 500Mi, cpu request to 200m.
- 0.15.0:
- Added dedot processor - auto replace
.
in log field to_
.
- Added dedot processor - auto replace
- 0.14.0:
- Fix typo in
fargateLogRouter
- Fix typo in
- 0.13.0:
- Removal of field
log_type
. Auto populatingtype
instead.
- Removal of field
- 0.12.0:
- Added auto detection for log_level field.
- 0.11.0:
- Upgrade image
logzio/logzio-fluentd:1.2.0
:- Upgrade to
fluentd 1.15
. - Upgrade plugin
fluent-plugin-kubernetes_metadata_filter
to3.1.2
.
- Upgrade to
- Upgrade image
- 0.10.0:
- Added an option to parse
log_type
annotation intolog_type
field.
- Added an option to parse
- 0.9.0:
- Added a default value for
env_id
field.
- Added a default value for
- 0.8.0:
- Add ability to add environment id with
env_id
field.
- Add ability to add environment id with
- 0.7.0:
- Add ability to change the secret name with
secretName
. #133
- Add ability to change the secret name with
- 0.6.1:
- Fix bug for
extraConfig
(#114).
- Fix bug for
- 0.6.0:
- Added
daemonset.priorityClassName
andwindowsDaemonset.priorityClassName
.
- Added
- 0.5.0:
- Add support for
daemonset.affinity
value. - Add support for fargate logging.
- Add support for
- 0.4.1:
- Upgrade default image version to
logzio/logzio-fluentd:1.1.1
.
- Upgrade default image version to
- 0.4.0:
- Allow dynamically set the log type for the logs.
- 0.3.0:
- Added new value fields:
daemonset.excludeFluentdPath
,daemonset.extraExclude
,daemonset.containersPath
,configmap.customSources
,configmap.customFilters
. - Added support for windows containers.
- Added new value fields:
- 0.2.0:
- Added
daemonset.nodeSelector
.
- Added
- 0.1.0:
- Upgrade default image version to
logzio/logzio-fluentd:1.0.2
which also supports ARM architecture. - Deprecated variables:
daemonset.containerdRuntime
,configmap.kubernetesContainerd
. - Added
configmap.partialDocker
,configmap.partialContainerd
that concatenate logs that split due to large size (over 16k). To learn more go to the configuration table. - Added
daemonset.cri
to match the partial log config to the cluster's CRI. To learn more go to the configuration table.
- Upgrade default image version to
- 0.0.4:
- Refactor configmaps
- 0.0.3:
- Edit configmap template name
- 0.0.2:
- Fix templates name - allow dyncmically change it.
- 0.0.1:
- Initial release.