logstash-integration-kafka-10.12.0-iam

Logstash Kafka integration 10.12.0 with AWS MSK IAM authentication support.
logstash-plugins#126 will add this support to the official Logstash plugin, if that PR is merged please use the official plugin that comes with Logstash.

Built with the v10.12.0 Kafka plugin on Logstash 7.17.6. Works with Logstash 7.x and 8.x versions as well.

You can install it for example in a Docker image:

FROM docker.elastic.co/logstash/logstash:7.17.6

# install a modified Kafka integration plugin with AWS IAM auth
# https://github.com/logstash-plugins/logstash-integration-kafka/pull/126/files
COPY logstash-integration-kafka-10.12.0-iam.gem /usr/share/logstash/
RUN /usr/share/logstash/bin/logstash-plugin remove logstash-integration-kafka
RUN /usr/share/logstash/bin/logstash-plugin install /usr/share/logstash/logstash-integration-kafka-10.12.0-iam.gem

Usage as input/output:

kafka {
 topics => ["..."]
 bootstrap_servers => "..."
 security_protocol => "SASL_SSL"
 sasl_mechanism => "AWS_MSK_IAM"
 sasl_jaas_config => "software.amazon.msk.auth.iam.IAMLoginModule required;"
}

Note, that this build has sasl_client_callback_handler_class hardcoded to software.amazon.msk.auth.iam.IAMClientCallbackHandler if the sasl_mechanism is AWS_MSK_IAM, so this parameter is not required as opposed to the PR in the official repo: logstash-plugins#126